system

Abstract

We provide the system. [Solution] A means for acquiring image data of a target structure using an image acquisition device, A means for determining the deterioration status of a structure using artificial intelligence that analyzes the aforementioned image data, A means of identifying repair locations and prioritizing them based on the assessment results, A means for creating an optimal reinforcement plan based on the aforementioned priorities, A system including means for presenting the aforementioned reinforcement plan to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is to efficiently diagnose the deterioration of buildings and infrastructure in which aging is progressing and formulate an appropriate reinforcement plan. In particular, since these diagnoses and the creation of reinforcement plans require advanced expertise and significant costs, it has become a major issue for local governments and individual building owners with a shortage of engineers and budget constraints. Also, in the face of increasing disaster risks, prompt and accurate repairs are required, and a system for supporting this is needed.

Means for Solving the Problems

[0005] This invention provides a system that diagnoses target areas by collecting image data of buildings and infrastructure using an image acquisition device and automatically analyzing the deterioration status using AI technology. Furthermore, it includes a means to identify areas requiring repair based on the analysis results and automatically assign priorities. Based on these priorities, it also formulates an optimal reinforcement plan and presents it to the user. In addition, it enables the input of additional information from the user and includes a function to manage and utilize building diagnostic data for the entire region, thereby supporting rapid and accurate disaster response. In this way, it is possible to achieve effective maintenance support while resolving problems such as high costs and shortages of skilled personnel.

[0006] An "image acquisition device" is a device equipped with the function of capturing image data of an object and collecting and transmitting that data.

[0007] "Image data" refers to digital information that visually records the appearance and characteristics of an object.

[0008] "Artificial intelligence" is a technology that uses computer-based data analysis to automatically determine and analyze the state of an object.

[0009] "Deterioration status" refers to information indicating the state of damage or quality deterioration in the object in question.

[0010] "Areas requiring repair" refers to parts that need repair or reinforcement due to deterioration.

[0011] "Priority" is an indicator used to determine the order in which tasks or items are performed and their relative importance.

[0012] A "reinforcement plan" refers to a plan that includes the steps to be taken, the materials to be used, and the construction period for repairing deteriorated areas.

[0013] A "user" refers to a person who uses this system to diagnose the condition of buildings and infrastructure and to check repair plans.

[0014] A "database" is an information system that accumulates, manages, and enables retrieval and utilization of a large amount of information as needed.

[0015] A "disaster response plan" is a plan that summarizes action guidelines and preparatory matters for responding quickly and appropriately in the event of a disaster.

Brief Explanation of Drawings

[0016] ]> [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Embodiment for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system of this invention is designed to effectively diagnose the deterioration of buildings and infrastructure and to provide an optimal reinforcement plan. This system mainly consists of an image acquisition device, a server, a terminal, and an interface with the user.

[0038] Users capture images of buildings and infrastructure using image acquisition devices such as smartphones. This image data includes the exterior and interior conditions of buildings and contains important information. The device transmits this image data to a server for further analysis.

[0039] The server analyzes the received image data using AI technology. Specifically, the AI ​​model automatically analyzes the images and identifies specific degradation patterns. In this process, the AI ​​identifies specific anomalies such as cracks, corrosion, and rust, and quantifies the degree of degradation.

[0040] After the analysis to assess the degree of deterioration is complete, the server identifies areas that require repair. Furthermore, the AI ​​automatically determines the repair priority based on the severity and scope of the deterioration, as well as the user's budget and priorities.

[0041] Based on these priorities, the server creates an optimal reinforcement plan for the user. This plan includes the necessary steps, materials, and schedule for repairing deterioration, and is presented in a format that is easy for the user to understand.

[0042] As a concrete example, suppose a local government uses this system to structurally evaluate aging bridges within its area. The user, a local government employee, photographs various parts of the bridge and sends the images to the server. The server uses AI to analyze the images, detecting deterioration of support pillars and cable wear. Based on the results, areas requiring urgent repair are identified, and an optimal repair schedule and budget proposal are provided to the local government. This process not only significantly improves the accuracy and efficiency of repairs but also helps to effectively utilize limited resources.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure to be diagnosed. Once the capture is complete, the device prepares the image data and gets ready to send it to the server.

[0046] Step 2:

[0047] The device compresses the image data captured by the user, adds metadata (e.g., date and time of capture and location information), and sends it to the server via a secure communication protocol.

[0048] Step 3:

[0049] The server temporarily stores the received image data in storage and converts the images into a format that can be used for AI analysis. Specifically, it adjusts the image size and removes noise, preparing the images for feature extraction.

[0050] Step 4:

[0051] The server inputs the formatted image data into an AI model to analyze the deterioration of buildings and infrastructure. The AI ​​model identifies and quantitatively evaluates anomalies such as cracks, corrosion, and rust. Based on this output, the degree and extent of deterioration are determined.

[0052] Step 5:

[0053] The server identifies areas requiring repair based on analysis results obtained by AI. Furthermore, it automatically determines the priority of repair work by considering multiple factors (severity of deterioration, budget, priorities, etc.).

[0054] Step 6:

[0055] The server creates an optimal reinforcement plan based on prioritized information. This plan is designed to include specific repair steps, materials to be used, and a schedule.

[0056] Step 7:

[0057] The terminal presents the user with an augmentation plan created by the server. The user can review the augmentation plan and send feedback to the server as needed.

[0058] Step 8:

[0059] The server analyzes user feedback and additional information, updating reinforcement plans as needed. Ultimately, it can be linked with a regional information database to store repair data and use it for disaster response.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] There is a need to diagnose the deterioration of buildings and infrastructure early and accurately, and to provide efficient and optimal reinforcement plans. However, conventional methods have limitations in the accuracy and efficiency of identifying deterioration and formulating repair plans, and there are challenges in appropriately allocating resources.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for collecting image information of a target facility using an image acquisition function, means for determining the extent of damage to the facility using artificial intelligence that analyzes the image information, and means for identifying repair locations and setting their importance based on the determination results. This makes it possible to provide efficient and highly accurate deterioration diagnosis and repair plans.

[0065] The "image acquisition function" is a function used to capture the exterior and interior condition of the target facility, and primarily collects information using image sensors.

[0066] "Image information" refers to digital information that indicates the condition of the target facility, and this includes data acquired as photographs and videos.

[0067] "Artificial intelligence" is a technology that analyzes large amounts of data to identify specific patterns or anomalies, and in this invention, it is used in particular to determine the state of deterioration.

[0068] "Means for determining the extent of damage" refers to methods for analyzing acquired image information to evaluate the condition of the target facility and identify damaged or deteriorated areas.

[0069] "Methods for identifying repair locations" refer to methods that, based on the results of artificial intelligence analysis, clearly indicate the specific areas that require repair.

[0070] A "means for setting importance" refers to a method for automatically determining the priority of repairs, taking into account the severity and scope of the damage.

[0071] A "repair plan" is a plan that includes specific repair methods, resource allocation, and schedules for identified repair areas.

[0072] A "generative AI model" is an artificial intelligence model that generates information through natural language processing and image analysis to improve the accuracy of planning.

[0073] A "prompt statement" is an instruction statement used to convey requests to a generating AI model, and is used when a user customizes a repair plan.

[0074] This invention specifically describes a system for diagnosing the deterioration of buildings and infrastructure and providing appropriate repair plans. The system mainly consists of an image acquisition function, a server, a terminal, and a user interface.

[0075] Users collect image information of target structures using image acquisition devices such as smartphones and drones. These devices are equipped with high-precision cameras, enabling the collection of multifaceted and detailed data. This makes it possible to capture even small damages and deterioration that are not visible to the naked eye.

[0076] The terminal is responsible for transmitting the collected image information to the server. The data is properly compressed and transferred to the server quickly and securely. Compression techniques are used that do not compromise image quality and maintain data accuracy.

[0077] When the server receives image information, it performs analysis using artificial intelligence. Specifically, a generative AI model analyzes the image using a deep learning algorithm to automatically identify deterioration patterns such as cracks and corrosion. The server extracts features from the image and determines the extent of the damage by comparing them with an existing database.

[0078] Once the analysis is complete, the server identifies the repair locations based on the results and sets their importance level. The importance level is determined based on the severity and scope of the damage, as well as the budget and priorities provided by the user. During this process, users can input additional information and requests using prompts. For example, by telling the server, "This part is particularly important; please prioritize its repair schedule," a more precise repair plan can be generated.

[0079] As a concrete example, consider a case where a city hall employee inspects a local bridge. The employee takes detailed photographs of the aging parts of the bridge and sends the images to a server via a terminal. The server immediately analyzes the images and identifies deterioration of the bridge's supports and cables. Based on the analysis results, it identifies the most urgent repair areas and sets repair priorities. This information is provided to the employee in a user-friendly format, enabling quick and accurate decision-making.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The user uses an image acquisition device to capture images of the target structure. The captured images contain detailed information about the structure from multiple angles. The input is the physical structure, and the output is digital image data. The user selects the necessary images and performs the capture operation.

[0083] Step 2:

[0084] The terminal compresses and encrypts the collected image data before sending it to the server. This ensures the efficiency and security of the transfer. The input is the captured image data, and the output is the compressed and encrypted data. The terminal then performs the operation of transferring this data to the server at high speed.

[0085] Step 3:

[0086] The server decompresses the received compressed image data and prepares it for analysis. The input is encrypted compressed data, and the output is image data in a format suitable for analysis. After decompression, the server organizes the data for artificial intelligence analysis.

[0087] Step 4:

[0088] The server automatically identifies degradation patterns from image data using a generative AI model. This process employs deep learning algorithms to extract features from the images. The input is the decompressed image data, and the output is the identified degradation patterns and evaluation results. The server then analyzes this data and performs an evaluation of the structure's condition.

[0089] Step 5:

[0090] The server identifies and prioritizes repair locations based on the degradation assessment results. The AI ​​lists the necessary repair locations and determines their priority based on the severity and extent of the damage. The input is the degradation pattern assessment results, and the output is a list of prioritized repair locations. The server analyzes the data and selects the most important repair targets.

[0091] Step 6:

[0092] The server receives prompts from the user and makes adjustments to reflect them in the repair plan. Prompts allow the user to provide additional instructions or requests. The input is the user's prompts, and the output is the adjusted repair plan. The server considers the user's needs and flexibly modifies the plan.

[0093] Step 7:

[0094] The server creates and provides the user with a final repair plan. This plan includes specific repair steps, a list of necessary materials, and a schedule. The input is coordinated repair data and priority information, and the output is a user-friendly plan. The server organizes the data, constructs the plan, and presents it to the user.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] The deterioration of buildings and infrastructure, leading to reduced safety and increased repair costs due to the progression of deterioration, are significant challenges in the management of public and large-scale facilities. Conventional methods struggle to accurately assess the extent of deterioration and formulate efficient reinforcement plans. Especially in large-scale facilities, constant monitoring is required, necessitating real-time information gathering and feedback of the results.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for acquiring image data of a target facility using an image acquisition device, means for determining the deterioration status of the facility using artificial intelligence that analyzes the image data, and means for notifying abnormalities in real time using monitoring equipment. This enables real-time monitoring of facility deterioration, allowing for rapid response and the formulation of efficient reinforcement plans.

[0100] An "image acquisition device" is a device used to acquire image data of a target facility, and includes photographic equipment such as cameras.

[0101] "Target facilities" is a general term for buildings and infrastructure that are subject to deterioration assessment.

[0102] "Image data" refers to digital data containing visual information acquired by an image acquisition device.

[0103] "Artificial intelligence" refers to a computer program or algorithm used to analyze image data and determine its degree of deterioration.

[0104] "Discrimination results" refer to data indicating the judgment regarding the degree of deterioration obtained through analysis by artificial intelligence.

[0105] "Areas requiring repair" refers to parts or areas of a facility that require restoration or reinforcement, as identified based on the assessment results.

[0106] "Priority" refers to a ranking assigned to determine the order in which work is carried out when repairing or reinforcing areas.

[0107] A "reinforcement plan" is a plan that is constructed based on the degree of deterioration and includes the repair methods, materials to be used, and schedule for the areas that need repair.

[0108] "Users" refers to those who receive a presentation of the reinforcement plan, including facility managers and owners.

[0109] "Monitoring equipment" refers to devices used to patrol the interior of a facility and diagnose deterioration in real time, and typically includes mobile robots.

[0110] "Communication equipment" refers to devices used by users to send and receive data, and includes smartphones and computers.

[0111] The system for implementing this invention is configured as follows.

[0112] First, the user uses an image acquisition device to capture images of various parts of the target facility. High-resolution cameras mounted on smartphones or monitoring equipment are used as image acquisition devices, allowing for efficient capture of deteriorated areas of the target facility. The captured image data is then transmitted to a server via a communication device.

[0113] The server analyzes the received image data using artificial intelligence. This analysis utilizes deep learning frameworks such as TENSORFLOW® and PyTorch, and the generative AI model automatically identifies degradation patterns. Specifically, the data processing involves pre-processing the images to identify specific degradation forms such as cracks and corrosion, and scoring their severity.

[0114] Next, the server identifies areas requiring repair based on the assessment results and assigns a priority to each area. Prioritization is determined based on the severity and scope of deterioration, as well as the user's budget and perceived importance. Based on this, an optimal reinforcement plan is created and presented to the user as documentation. This reinforcement plan includes the necessary processes, materials to be used, and construction schedule. Users can input feedback and additional requests regarding this plan using a communication device. This information is also processed by the server and incorporated into the plan.

[0115] As a concrete example, if a monitoring device detects a crack in a wall during a patrol at a commercial facility, the data is immediately sent to a server for analysis, after which repair work for the following day is planned. An example of a prompt message to the generated AI model would be, "Diagnose the deterioration status of the facility from the images captured by the camera and generate the necessary repair plan." The introduction of this system will improve the safety and efficiency of facility management.

[0116] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0117] Step 1:

[0118] The user uses an image acquisition device to acquire images of the target facility. The input is images of the facility's exterior and specific interior areas, generating high-resolution image data. The output is this acquired image data.

[0119] Step 2:

[0120] The user uses a communication device to send acquired image data to the server. The input is high-resolution image data, and the output is image data securely transmitted to the server. In this step, data transfer is performed using a communication protocol.

[0121] Step 3:

[0122] The server inputs the received image data into the AI ​​analysis system. The input is the transmitted image data, and the output is the analysis result of the degradation patterns contained in the image. As part of the data processing, image preprocessing is performed, and patterns such as cracks and corrosion are identified using a generated AI model.

[0123] Step 4:

[0124] Based on the analysis results, the server uses AI to identify areas requiring repair and their priority. The input is analyzed degradation data, and the output is a list of degraded areas and their priorities. A ranking is generated using a prioritization algorithm based on the severity of the degradation.

[0125] Step 5:

[0126] The server generates a reinforcement plan based on identified repair locations and their priorities. The input is a list of prioritized repair locations, and the output is a detailed reinforcement plan. The plan includes materials to be used, work processes, and schedules, and an optimization algorithm is employed.

[0127] Step 6:

[0128] The server presents the generated augmentation plan to the user. The input is the created augmentation plan, and the output is information about the augmentation plan displayed in an easy-to-understand format. Details of the plan are visualized through the user interface.

[0129] Step 7:

[0130] Users input additional information regarding reinforcement plans via a communication device and send it to the server. The input consists of supplementary information and requested modifications, while the output is updated plan information. This allows for flexible plan adjustments.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention combines a system that diagnoses the deterioration status of buildings and infrastructure and provides an optimal reinforcement plan with an emotion engine that recognizes the user's emotions. This system mainly consists of an image acquisition device, a server, a terminal, and an emotion engine.

[0133] The user takes a picture of the target structure using an image acquisition device such as a smartphone and sends the image data to the terminal. The terminal sends the image data to a server, which analyzes the received image using an AI model. The AI ​​identifies specific deterioration patterns from the image and quantifies the degree of deterioration.

[0134] Next, the server identifies areas requiring repair based on its degradation status and prioritizes repairs by considering multiple factors. Based on this, an optimal reinforcement plan is formulated, and the emotion engine further analyzes user reactions.

[0135] The emotion engine analyzes the user's voice and facial expressions as they review the reinforcement plan, and evaluates their emotions. This information is then used to suggest additional information and support that alleviate the user's anxiety and concerns. For example, if the emotion engine senses that the user is anxious, it can provide detailed explanations and present actual repair examples to address that anxiety.

[0136] As a concrete example, suppose an individual user uses this system to inspect the old roof of their home. The user takes photos of the roof and sends them to the server, where the AI ​​identifies deteriorated areas. The server evaluates the extent of the deterioration and creates a reinforcement plan for areas that need repair. If the user feels uneasy after reviewing the plan, the emotion engine detects this and presents successful repair case studies by roofing professionals to provide reassurance. In this way, user satisfaction can be increased by providing an integrated solution that combines technical problem diagnosis with emotional support.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure they want to diagnose. After taking the pictures, the image data is saved to the device.

[0140] Step 2:

[0141] The terminal compresses the stored image data, adds the necessary metadata, and sends it to the server. A secure communication protocol is used for transmission to avoid data loss.

[0142] Step 3:

[0143] The server preprocesses the received image data before inputting it into the analysis system. This preprocessing includes resizing and noise reduction, preparing the images for analysis.

[0144] Step 4:

[0145] The server inputs pre-processed image data into an AI model to analyze the degradation status. The AI ​​model identifies anomalies in the image (e.g., cracks and rust), quantifies them, and generates an evaluation result.

[0146] Step 5:

[0147] The server identifies areas requiring repair based on evaluation results obtained from the AI ​​model. Next, it considers multiple factors such as the degree of deterioration, the importance of the structure, and the user's budget to determine the priority of repairs.

[0148] Step 6:

[0149] The server creates a detailed reinforcement plan based on the prioritized results. This plan includes repair processes, materials to be used, and implementation schedules, and is compiled in an easily viewable format such as PDF or images.

[0150] Step 7:

[0151] When the terminal presents the augmentation plan received from the server to the user, the emotion engine is also activated. When the user reviews the augmentation plan, the emotion engine performs voice analysis and facial recognition to detect the user's emotions.

[0152] Step 8:

[0153] Based on the user's emotional information obtained by the emotion engine, the server provides additional information to alleviate the user's anxiety and concerns about the plan. For example, if anxiety is detected, it may present past success stories or provide more detailed explanations.

[0154] Step 9:

[0155] After receiving additional information and support, users will be able to enter questions and additional instructions regarding the reinforcement plan into the terminal. The terminal will collect this information and send it to the server in real time.

[0156] Step 10:

[0157] The server analyzes additional information received from the user and updates the reinforcement plan if necessary. The updated plan is then sent to the terminal and presented to the user, optimizing the plan.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0160] Conventional building and infrastructure deterioration assessment systems focused on technical assessments and the presentation of reinforcement plans, but lacked sufficient support that considered the user's psychological state. As a result, user anxieties and concerns about the proposed reinforcement plans were not addressed, leading to low user satisfaction with the assessment results and proposals. Furthermore, efficient information management that takes into account region-specific conditions remained a challenge.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] In this invention, the server includes means for determining the deterioration status using artificial intelligence to analyze image data, means for creating and presenting reinforcement plans, emotion engine means for detecting user emotions and providing appropriate additional information, and means for analyzing user emotion data and proposing psychological support. This makes it possible to not only provide technical diagnostic results but also to provide comprehensive support that takes user emotions into consideration, thereby improving user satisfaction with deterioration diagnosis and reinforcement proposals.

[0163] An "image acquisition device" refers to equipment used to acquire image data of structures such as buildings and infrastructure. Specifically, this includes smartphones and digital cameras equipped with camera functions.

[0164] "Artificial intelligence" is a technology developed to allow computer systems to mimic human intelligence, and is primarily used for data analysis and pattern recognition.

[0165] "Deterioration status" refers to the condition or degree to which a structure has deteriorated due to aging or environmental factors, and whether repairs are necessary.

[0166] A "reinforcement plan" is a plan that outlines specific procedures and methods for repairing or improving deteriorated structures, and also includes their priorities.

[0167] An "emotion engine" refers to a technology or system that analyzes a user's voice and facial expressions to recognize their emotions and provide appropriate support or additional information.

[0168] "Prioritizing" means evaluating the importance and necessity of multiple options or tasks, and deciding the order in which to perform them.

[0169] "Psychological support" refers to activities and information provided that take emotional aspects into consideration in order to alleviate users' anxiety and concerns and provide them with a sense of security.

[0170] "Local information" refers to geographical, climatic, social, and economic data and knowledge related to a specific region, and is used for disaster response and planning.

[0171] This system consists of an image acquisition device, a server, a terminal, and an emotion engine. First, the user acquires image data of buildings and infrastructure using an image acquisition device such as a smartphone or digital camera. The image data captured by the user is sent to a terminal, such as a smartphone or personal computer. The terminal then sends this image data to the server.

[0172] The server possesses powerful computing capabilities and analyzes the received image data based on a generative AI model. This generative AI model is built using deep learning techniques to identify deterioration patterns from building images. Typical frameworks used include TensorFlow and PyTorch. The server detects signs of deterioration in the images and quantifies the degree of deterioration.

[0173] Furthermore, based on the analysis results, the server identifies where repairs are needed, prioritizes them, and creates an optimal reinforcement plan. This plan is presented to the user via their terminal. For example, it utilizes data on the extent and location of cracks in the building to specifically show effective reinforcement methods and procedures.

[0174] Subsequently, the user reviews the reinforcement plan using a device. During this process, the device captures the user's facial expressions and voice data, which are then analyzed by a server using an emotion engine. This emotion engine has the technology to identify whether the user has any anxieties or doubts about the plan. Based on the detected emotions, additional information is provided to alleviate the user's anxiety. For example, actual examples of successful reinforcements and testimonials from construction companies may be presented.

[0175] As a concrete example, consider a case where a user inspects the old roof of their home. The user takes a picture of the roof with their smartphone and sends it to a server for analysis. The server identifies the deteriorated areas, develops a reinforcement plan, and presents it. If the emotion engine detects the user's anxiety, it will show past repair examples from roofing experts to reassure the user.

[0176] An example of a prompt message input to a generated AI model is: "This program combines an AI system that diagnoses the deterioration status of a building with an emotion engine that recognizes user emotions. Based on the information obtained from image data, identify deteriorated areas, develop a reinforcement plan, and provide user support through emotion analysis."

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The user uses an image acquisition device to capture image data of the target building or infrastructure. Specifically, they use the camera function of their smartphone to take multiple photos of the area to be diagnosed. The captured image data is saved to the smartphone's memory. The input here is optical information indicating the current state of the building, and the output is data in an image file format (such as JPEG or PNG).

[0180] Step 2:

[0181] The terminal uploads image data acquired from the user's smartphone to the server. In this process, the terminal transmits data to the server using the internet. Specifically, the user selects an image using a dedicated application and presses the send button. The input is image data stored on the terminal, and the output is image data stored on the server's storage.

[0182] Step 3:

[0183] The server analyzes the received image data using a generative AI model. The AI ​​model is trained to identify degradation patterns (cracks, distortion, discoloration, etc.). Specifically, the AI ​​analyzes the pixel information of the image, extracts features, and performs pattern recognition. The input is image data stored on the server, and the output is quantified data indicating the areas of degradation (e.g., crack width, location).

[0184] Step 4:

[0185] The server creates a prioritized reinforcement plan based on the degradation data obtained from the analysis. The server determines the priority of repairs, taking into account the severity and scope of the degradation. Specifically, it analyzes quantitative data and generates repair proposals in accordance with a reinforcement plan template. The input is quantitative data on the degraded areas, and the output is a prioritized reinforcement plan.

[0186] Step 5:

[0187] The server sends the generated augmentation plan to the user via the terminal. The user reviews and understands the augmentation plan on the terminal. Specifically, the user opens the application and views the plan file. The input is the augmentation plan (digital document), and the output is the augmentation plan provided to the user as visual information.

[0188] Step 6:

[0189] The terminal acquires user voice and facial expression data when reviewing the reinforcement plan and sends it to the server. During this process, the terminal uses its camera and microphone to record the user's reactions. The input is the user's voice and video, and the output is emotion data sent to the server.

[0190] Step 7:

[0191] The server uses an emotion engine to analyze the user's voice and facial expressions and perform an emotional assessment. The server recognizes whether the user is experiencing anxiety or concern and generates appropriate additional information. The input is emotional data, and the output is additional information designed to enhance the user's sense of security (e.g., success stories, expert comments).

[0192] Step 8:

[0193] The server provides the user with additional information based on sentiment analysis via the terminal. The user uses this information to deepen their understanding of the reinforcement plan and reduce anxiety. Specifically, the user reviews additional information such as videos and text. The input is supplementary information based on sentiment analysis, and the output is an improvement in the user's psychological sense of security.

[0194] (Application Example 2)

[0195] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0196] In diagnosing deterioration of factory equipment and infrastructure, conventional methods make it difficult to accurately determine the extent of deterioration and formulate reinforcement plans based on the results. Furthermore, there is a lack of care that addresses the anxieties and concerns felt by users. This can lead to delays in timely and appropriate maintenance, potentially reducing work efficiency.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0198] In this invention, the server includes means for acquiring image information of an object using an image acquisition device, means for identifying the deterioration status of the object using an automatic processing device that analyzes the image information, means for identifying repair locations and setting their importance based on the identification results, means for creating an optimal reinforcement plan based on the importance, means for presenting the reinforcement plan to the user, and means for detecting the user's emotions and providing information in accordance with those emotions. This makes it possible to accurately diagnose the deterioration of factory equipment and optimize maintenance plans that take into account the user's emotions.

[0199] An "image acquisition device" is a device used to acquire image information of an object.

[0200] An "automated processing device" is a system that analyzes acquired image information to identify the degree of deterioration of an object.

[0201] "Importance" is a measure of priority assigned to the identified repair locations.

[0202] A "reinforcement plan" is a plan for optimized repair and reinforcement based on the identified deterioration status.

[0203] A "user" is a person or organization that operates the system and receives suggestions for reinforcement plans.

[0204] "Providing information tailored to emotions" refers to detecting the user's emotions and then providing additional information to alleviate their anxiety or concerns.

[0205] To implement this invention, first, a camera device capable of photographing equipment and structures within a factory is used as the image acquisition device. The user uses this camera to photograph objects that may be deteriorating. The captured image data is transmitted to the server via a terminal.

[0206] The server uses an automated processing unit, specifically an AI processing module, to analyze the transmitted image data. Through this analysis, the AI ​​system identifies the deterioration patterns of the object and quantifies the degree of deterioration. For this AI processing, for example, open-source image processing libraries such as OpenCV or deep learning frameworks such as TensorFlow can be used.

[0207] Furthermore, the server assigns a priority level to the identified repair locations and creates an optimal reinforcement plan based on this level. This reinforcement plan is created using planning software to develop a feasible maintenance plan tailored to the degree and scale of the identified deterioration.

[0208] Meanwhile, the server uses emotion recognition software to analyze the user's response to the reinforcement plan. This is done by evaluating facial expressions and voice acquired through a webcam and microphone. This emotion evaluation uses an emotion analysis engine such as EmotionEngine. Based on the results, if the user feels anxiety or concern, additional information and specific examples can be provided to support the user according to their needs.

[0209] As a concrete example, consider the case of performing regular inspections of factory equipment. When a user takes photos of the equipment and sends them to the system, the server first uses an AI model to diagnose the degree of deterioration and design a repair plan. At the same time, an emotion engine detects the user's reactions and, if necessary, presents additional information or past success stories, thereby improving the sense of security during the inspection work.

[0210] An example of a prompt to input into the generating AI model is as follows: "Please tell me how to effectively implement a degradation diagnosis support application for use in a factory. In particular, please focus on optimizing information delivery based on the worker's emotions."

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] The user uses a camera device to photograph equipment and structures within the factory and acquires the image data. This image data is transmitted to the server via a terminal. The input is the captured image data, and the output is the transmitted image data.

[0214] Step 2:

[0215] The server uses an AI model to analyze the received image data. Specifically, it uses OpenCV and TensorFlow to perform image processing, identify degradation patterns, and perform classification. The input is the image data sent in step 1, and the output is the identified degradation patterns and classification results.

[0216] Step 3:

[0217] The server sets the importance of the repair locations based on the identified degradation information. Using planning software, it lists the locations in descending order of priority, taking into account the degree of degradation. The input is the identification result from step 2, and the output is a list of repair locations with assigned priorities.

[0218] Step 4:

[0219] The server creates an optimal reinforcement plan based on a list of repair locations. This involves using a planning algorithm to develop specific maintenance procedures and schedules. The input is the priority list from step 3, and the output is a detailed reinforcement plan.

[0220] Step 5:

[0221] The server uses emotion recognition software to analyze the user's facial expressions and voice when presenting reinforcement plans. It uses emotion analysis tools such as EmotionEngine to evaluate emotions like anxiety and concern. The input is user facial expression and voice data, and the output is the result of the emotion evaluation.

[0222] Step 6:

[0223] Based on the emotional assessment results, the server provides the user with additional information and success stories to help alleviate anxiety. This helps the user accept the reinforcement plan with confidence. The input is the emotional assessment result from step 5, and the output is the additional information and success stories presented to the user.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] The system of this invention is designed to effectively diagnose the deterioration of buildings and infrastructure and to provide an optimal reinforcement plan. This system mainly consists of an image acquisition device, a server, a terminal, and an interface with the user.

[0241] Users capture images of buildings and infrastructure using image acquisition devices such as smartphones. This image data includes the exterior and interior conditions of buildings and contains important information. The device transmits this image data to a server for further analysis.

[0242] The server analyzes the received image data using AI technology. Specifically, the AI ​​model automatically analyzes the images and identifies specific degradation patterns. In this process, the AI ​​identifies specific anomalies such as cracks, corrosion, and rust, and quantifies the degree of degradation.

[0243] After the analysis to assess the degree of deterioration is complete, the server identifies areas that require repair. Furthermore, the AI ​​automatically determines the repair priority based on the severity and scope of the deterioration, as well as the user's budget and priorities.

[0244] Based on these priorities, the server creates an optimal reinforcement plan for the user. This plan includes the necessary steps, materials, and schedule for repairing deterioration, and is presented in a format that is easy for the user to understand.

[0245] As a concrete example, suppose a local government uses this system to structurally evaluate aging bridges within its area. The user, a local government employee, photographs various parts of the bridge and sends the images to the server. The server uses AI to analyze the images, detecting deterioration of support pillars and cable wear. Based on the results, areas requiring urgent repair are identified, and an optimal repair schedule and budget proposal are provided to the local government. This process not only significantly improves the accuracy and efficiency of repairs but also helps to effectively utilize limited resources.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure to be diagnosed. Once the capture is complete, the device prepares the image data and gets ready to send it to the server.

[0249] Step 2:

[0250] The device compresses the image data captured by the user, adds metadata (e.g., date and time of capture and location information), and sends it to the server via a secure communication protocol.

[0251] Step 3:

[0252] The server temporarily stores the received image data in storage and converts the images into a format that can be used for AI analysis. Specifically, it adjusts the image size and removes noise, preparing the images for feature extraction.

[0253] Step 4:

[0254] The server inputs the formatted image data into an AI model to analyze the deterioration of buildings and infrastructure. The AI ​​model identifies and quantitatively evaluates anomalies such as cracks, corrosion, and rust. Based on this output, the degree and extent of deterioration are determined.

[0255] Step 5:

[0256] The server identifies areas requiring repair based on analysis results obtained by AI. Furthermore, it automatically determines the priority of repair work by considering multiple factors (severity of deterioration, budget, priorities, etc.).

[0257] Step 6:

[0258] The server creates an optimal reinforcement plan based on prioritized information. This plan is designed to include specific repair steps, materials to be used, and a schedule.

[0259] Step 7:

[0260] The terminal presents the user with an augmentation plan created by the server. The user can review the augmentation plan and send feedback to the server as needed.

[0261] Step 8:

[0262] The server analyzes user feedback and additional information, updating reinforcement plans as needed. Ultimately, it can be linked with a regional information database to store repair data and use it for disaster response.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] There is a need to diagnose the deterioration of buildings and infrastructure early and accurately, and to provide efficient and optimal reinforcement plans. However, conventional methods have limitations in the accuracy and efficiency of identifying deterioration and formulating repair plans, and there are challenges in appropriately allocating resources.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for collecting image information of a target facility using an image acquisition function, means for determining the extent of damage to the facility using artificial intelligence that analyzes the image information, and means for identifying repair locations and setting their importance based on the determination results. This makes it possible to provide efficient and highly accurate deterioration diagnosis and repair plans.

[0268] The "image acquisition function" is a function used to capture the exterior and interior condition of the target facility, and primarily collects information using image sensors.

[0269] "Image information" refers to digital information that indicates the condition of the target facility, and this includes data acquired as photographs and videos.

[0270] "Artificial intelligence" is a technology that analyzes large amounts of data to identify specific patterns or anomalies, and in this invention, it is used in particular to determine the state of deterioration.

[0271] "Means for determining the extent of damage" refers to methods for analyzing acquired image information to evaluate the condition of the target facility and identify damaged or deteriorated areas.

[0272] "Methods for identifying repair locations" refer to methods that, based on the results of artificial intelligence analysis, clearly indicate the specific areas that require repair.

[0273] A "means for setting importance" refers to a method for automatically determining the priority of repairs, taking into account the severity and scope of the damage.

[0274] A "repair plan" is a plan that includes specific repair methods, resource allocation, and schedules for identified repair areas.

[0275] A "generative AI model" is an artificial intelligence model that generates information through natural language processing and image analysis to improve the accuracy of planning.

[0276] A "prompt statement" is an instruction statement used to convey requests to a generating AI model, and is used when a user customizes a repair plan.

[0277] This invention specifically describes a system for diagnosing the deterioration of buildings and infrastructure and providing appropriate repair plans. The system mainly consists of an image acquisition function, a server, a terminal, and a user interface.

[0278] Users collect image information of target structures using image acquisition devices such as smartphones and drones. These devices are equipped with high-precision cameras, enabling the collection of multifaceted and detailed data. This makes it possible to capture even small damages and deterioration that are not visible to the naked eye.

[0279] The terminal is responsible for transmitting the collected image information to the server. The data is properly compressed and transferred to the server quickly and securely. Compression techniques are used that do not compromise image quality and maintain data accuracy.

[0280] When the server receives the image information, it performs analysis using artificial intelligence. Specifically, the generative AI model analyzes the image using a deep learning algorithm to automatically identify deterioration patterns such as cracks and corrosion. The server extracts features from the image and determines the damage status by comparing it with an existing database.

[0281] When the analysis is completed, the server identifies the repair location based on the discrimination result and sets the importance level. The importance level is set based on the severity and scope of the damage, as well as the budget and priorities presented by the user. In this process, using prompt sentences, the user can input additional information and requests. For example, by conveying a request such as "This part is particularly important. Please prioritize the repair schedule." to the server, a more precise repair plan is generated.

[0282] As a specific example, consider the case where a city hall staff member diagnoses a local bridge. The staff member takes detailed pictures of the deteriorated parts of the bridge and transmits the images to the server via a terminal. The server immediately analyzes the images and identifies the deterioration of the bridge's columns and cables. Based on the analysis results, the most urgent repair locations are identified and the repair priorities are set. Since this information is provided to the staff in a user-friendly format, quick and accurate decision-making is possible.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user uses an image acquisition device to take pictures of the target structure. The taken images contain detailed information from multiple angles of the structure. The input is a physical structure, and the output is digital-form image data. The user performs the operation of selecting and taking the necessary images.

[0286] Step 2:

[0287] The terminal compresses and encrypts the collected image data for transmitting it to the server. This ensures the efficiency and security of the transfer. The input is the captured image data, and the output is the compressed and encrypted data. The terminal performs the operation of transferring this data to the server at high speed.

[0288] Step 3:

[0289] The server decompresses the received compressed image data and prepares for analysis. The input is the encrypted compressed data, and the output is the image data in a format suitable for analysis. After decompression, the server performs the operation of organizing the data for artificial intelligence analysis.

[0290] Step 4:

[0291] The server automatically identifies the degradation pattern from the image data using the generated AI model. In this process, deep learning algorithms are used and feature extraction is performed from the image. The input is the decompressed image data, and the output is the identified degradation pattern and evaluation result. The server performs the operation of analyzing these data and evaluating the state of the structure.

[0292] Step 5:

[0293] Based on the evaluation result of the degradation, the server identifies the repair locations and assigns priorities. The AI lists up the necessary repair locations and determines the priorities based on the severity and the affected range of the damage. The input is the evaluation result of the degradation pattern, and the output is a list of the prioritized repair locations. The server performs the operation of analyzing the data and selecting the most important repair target.

[0294] Step 6:

[0295] The server receives prompts from the user and makes adjustments to reflect them in the repair plan. Prompts allow the user to provide additional instructions or requests. The input is the user's prompts, and the output is the adjusted repair plan. The server considers the user's needs and flexibly modifies the plan.

[0296] Step 7:

[0297] The server creates and provides the user with a final repair plan. This plan includes specific repair steps, a list of necessary materials, and a schedule. The input is coordinated repair data and priority information, and the output is a user-friendly plan. The server organizes the data, constructs the plan, and presents it to the user.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] The deterioration of buildings and infrastructure, leading to reduced safety and increased repair costs due to the progression of deterioration, are significant challenges in the management of public and large-scale facilities. Conventional methods struggle to accurately assess the extent of deterioration and formulate efficient reinforcement plans. Especially in large-scale facilities, constant monitoring is required, necessitating real-time information gathering and feedback of the results.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for acquiring image data of a target facility using an image acquisition device, means for determining the deterioration status of the facility using artificial intelligence to analyze the image data, and means for notifying abnormalities in real time using monitoring equipment. As a result, it becomes possible to monitor the deterioration of the facility in real time and formulate a prompt response and an efficient reinforcement plan.

[0303] The "image acquisition device" is a device for acquiring image data of a target facility and includes imaging equipment such as a camera.

[0304] The "target facility" is a general term for buildings and infrastructure that are the targets for diagnosing the deterioration status.

[0305] The "image data" is digital data including visual information acquired by an image acquisition device.

[0306] The "artificial intelligence" is a computer program or algorithm used to analyze image data and determine the deterioration status.

[0307] The "discrimination result" is data indicating the judgment content regarding the deterioration status obtained by the analysis using artificial intelligence.

[0308] The "repair location" refers to the parts or areas of the facility that need to be repaired or reinforced, specified based on the discrimination result.

[0309] The "priority" is a ranking given to determine the work order when repairing or reinforcing the repair location.

[0310] The "reinforcement plan" is a plan constructed based on the deterioration status and including the repair method, materials used, schedule, etc. of the repair location.

[0311] The "user" is a person who receives the presentation of the reinforcement plan and includes the administrator and owner of the facility.

[0312] "Monitoring equipment" refers to devices used to patrol the interior of a facility and diagnose deterioration in real time, and typically includes mobile robots.

[0313] "Communication equipment" refers to devices used by users to send and receive data, and includes smartphones and computers.

[0314] The system for implementing this invention is configured as follows.

[0315] First, the user uses an image acquisition device to capture images of various parts of the target facility. High-resolution cameras mounted on smartphones or monitoring equipment are used as image acquisition devices, allowing for efficient capture of deteriorated areas of the target facility. The captured image data is then transmitted to a server via a communication device.

[0316] The server analyzes the received image data using artificial intelligence. This analysis utilizes deep learning frameworks such as TensorFlow and PyTorch, and the generative AI model automatically identifies degradation patterns. Specifically, the data processing involves pre-processing the images to identify specific degradation forms such as cracks and corrosion, and then scoring their severity.

[0317] Next, the server identifies areas requiring repair based on the assessment results and assigns a priority to each area. Prioritization is determined based on the severity and scope of deterioration, as well as the user's budget and perceived importance. Based on this, an optimal reinforcement plan is created and presented to the user as documentation. This reinforcement plan includes the necessary processes, materials to be used, and construction schedule. Users can input feedback and additional requests regarding this plan using a communication device. This information is also processed by the server and incorporated into the plan.

[0318] As a concrete example, if a monitoring device detects a crack in a wall during a patrol at a commercial facility, the data is immediately sent to a server for analysis, after which repair work for the following day is planned. An example of a prompt message to the generated AI model would be, "Diagnose the deterioration status of the facility from the images captured by the camera and generate the necessary repair plan." The introduction of this system will improve the safety and efficiency of facility management.

[0319] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0320] Step 1:

[0321] The user uses an image acquisition device to acquire images of the target facility. The input is images of the facility's exterior and specific interior areas, generating high-resolution image data. The output is this acquired image data.

[0322] Step 2:

[0323] The user uses a communication device to send acquired image data to the server. The input is high-resolution image data, and the output is image data securely transmitted to the server. In this step, data transfer is performed using a communication protocol.

[0324] Step 3:

[0325] The server inputs the received image data into the AI ​​analysis system. The input is the transmitted image data, and the output is the analysis result of the degradation patterns contained in the image. As part of the data processing, image preprocessing is performed, and patterns such as cracks and corrosion are identified using a generated AI model.

[0326] Step 4:

[0327] Based on the analysis results, the server uses AI to identify areas requiring repair and their priority. The input is analyzed degradation data, and the output is a list of degraded areas and their priorities. A ranking is generated using a prioritization algorithm based on the severity of the degradation.

[0328] Step 5:

[0329] The server generates a reinforcement plan based on identified repair locations and their priorities. The input is a list of prioritized repair locations, and the output is a detailed reinforcement plan. The plan includes materials to be used, work processes, and schedules, and an optimization algorithm is employed.

[0330] Step 6:

[0331] The server presents the generated augmentation plan to the user. The input is the created augmentation plan, and the output is information about the augmentation plan displayed in an easy-to-understand format. Details of the plan are visualized through the user interface.

[0332] Step 7:

[0333] Users input additional information regarding reinforcement plans via a communication device and send it to the server. The input consists of supplementary information and requested modifications, while the output is updated plan information. This allows for flexible plan adjustments.

[0334] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0335] This invention combines a system that diagnoses the deterioration status of buildings and infrastructure and provides an optimal reinforcement plan with an emotion engine that recognizes the user's emotions. This system mainly consists of an image acquisition device, a server, a terminal, and an emotion engine.

[0336] The user takes a picture of the target structure using an image acquisition device such as a smartphone and sends the image data to the terminal. The terminal sends the image data to a server, which analyzes the received image using an AI model. The AI ​​identifies specific deterioration patterns from the image and quantifies the degree of deterioration.

[0337] Next, the server identifies areas requiring repair based on its degradation status and prioritizes repairs by considering multiple factors. Based on this, an optimal reinforcement plan is formulated, and the emotion engine further analyzes user reactions.

[0338] The emotion engine analyzes the user's voice and facial expressions as they review the reinforcement plan, and evaluates their emotions. This information is then used to suggest additional information and support that alleviate the user's anxiety and concerns. For example, if the emotion engine senses that the user is anxious, it can provide detailed explanations and present actual repair examples to address that anxiety.

[0339] As a concrete example, suppose an individual user uses this system to inspect the old roof of their home. The user takes photos of the roof and sends them to the server, where the AI ​​identifies deteriorated areas. The server evaluates the extent of the deterioration and creates a reinforcement plan for areas that need repair. If the user feels uneasy after reviewing the plan, the emotion engine detects this and presents successful repair case studies by roofing professionals to provide reassurance. In this way, user satisfaction can be increased by providing an integrated solution that combines technical problem diagnosis with emotional support.

[0340] The following describes the processing flow.

[0341] Step 1:

[0342] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure they want to diagnose. After taking the pictures, the image data is saved to the device.

[0343] Step 2:

[0344] The terminal compresses the stored image data, adds the necessary metadata, and sends it to the server. A secure communication protocol is used for transmission to avoid data loss.

[0345] Step 3:

[0346] The server preprocesses the received image data before inputting it into the analysis system. This preprocessing includes resizing and noise reduction, preparing the images for analysis.

[0347] Step 4:

[0348] The server inputs pre-processed image data into an AI model to analyze the degradation status. The AI ​​model identifies anomalies in the image (e.g., cracks and rust), quantifies them, and generates an evaluation result.

[0349] Step 5:

[0350] The server identifies areas requiring repair based on evaluation results obtained from the AI ​​model. Next, it considers multiple factors such as the degree of deterioration, the importance of the structure, and the user's budget to determine the priority of repairs.

[0351] Step 6:

[0352] The server creates a detailed reinforcement plan based on the prioritized results. This plan includes repair processes, materials to be used, and implementation schedules, and is compiled in an easily viewable format such as PDF or images.

[0353] Step 7:

[0354] When the terminal presents the augmentation plan received from the server to the user, the emotion engine is also activated. When the user reviews the augmentation plan, the emotion engine performs voice analysis and facial recognition to detect the user's emotions.

[0355] Step 8:

[0356] Based on the user's emotional information obtained by the emotion engine, the server provides additional information to alleviate the user's anxiety and concerns about the plan. For example, if anxiety is detected, it may present past success stories or provide more detailed explanations.

[0357] Step 9:

[0358] After receiving additional information and support, users will be able to enter questions and additional instructions regarding the reinforcement plan into the terminal. The terminal will collect this information and send it to the server in real time.

[0359] Step 10:

[0360] The server analyzes additional information received from the user and updates the reinforcement plan if necessary. The updated plan is then sent to the terminal and presented to the user, optimizing the plan.

[0361] (Example 2)

[0362] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0363] Conventional building and infrastructure deterioration assessment systems focused on technical assessments and the presentation of reinforcement plans, but lacked sufficient support that considered the user's psychological state. As a result, user anxieties and concerns about the proposed reinforcement plans were not addressed, leading to low user satisfaction with the assessment results and proposals. Furthermore, efficient information management that takes into account region-specific conditions remained a challenge.

[0364] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0365] In this invention, the server includes means for determining the deterioration status using artificial intelligence to analyze image data, means for creating and presenting reinforcement plans, emotion engine means for detecting user emotions and providing appropriate additional information, and means for analyzing user emotion data and proposing psychological support. This makes it possible to not only provide technical diagnostic results but also to provide comprehensive support that takes user emotions into consideration, thereby improving user satisfaction with deterioration diagnosis and reinforcement proposals.

[0366] An "image acquisition device" refers to equipment used to acquire image data of structures such as buildings and infrastructure. Specifically, this includes smartphones and digital cameras equipped with camera functions.

[0367] "Artificial intelligence" is a technology developed to allow computer systems to mimic human intelligence, and is primarily used for data analysis and pattern recognition.

[0368] "Deterioration status" refers to the condition or degree to which a structure has deteriorated due to aging or environmental factors, and whether repairs are necessary.

[0369] A "reinforcement plan" is a plan that outlines specific procedures and methods for repairing or improving deteriorated structures, and also includes their priorities.

[0370] An "emotion engine" refers to a technology or system that analyzes a user's voice and facial expressions to recognize their emotions and provide appropriate support or additional information.

[0371] "Prioritizing" means evaluating the importance and necessity of multiple options or tasks, and deciding the order in which to perform them.

[0372] "Psychological support" refers to activities and information provided that take emotional aspects into consideration in order to alleviate users' anxiety and concerns and provide them with a sense of security.

[0373] "Local information" refers to geographical, climatic, social, and economic data and knowledge related to a specific region, and is used for disaster response and planning.

[0374] This system consists of an image acquisition device, a server, a terminal, and an emotion engine. First, the user acquires image data of buildings and infrastructure using an image acquisition device such as a smartphone or digital camera. The image data captured by the user is sent to a terminal, such as a smartphone or personal computer. The terminal then sends this image data to the server.

[0375] The server possesses powerful computing capabilities and analyzes the received image data based on a generative AI model. This generative AI model is built using deep learning techniques to identify deterioration patterns from building images. Typical frameworks used include TensorFlow and PyTorch. The server detects signs of deterioration in the images and quantifies the degree of deterioration.

[0376] Furthermore, based on the analysis results, the server identifies where repairs are needed, prioritizes them, and creates an optimal reinforcement plan. This plan is presented to the user via their terminal. For example, it utilizes data on the extent and location of cracks in the building to specifically show effective reinforcement methods and procedures.

[0377] Subsequently, the user reviews the reinforcement plan using a device. During this process, the device captures the user's facial expressions and voice data, which are then analyzed by a server using an emotion engine. This emotion engine has the technology to identify whether the user has any anxieties or doubts about the plan. Based on the detected emotions, additional information is provided to alleviate the user's anxiety. For example, actual examples of successful reinforcements and testimonials from construction companies may be presented.

[0378] As a concrete example, consider a case where a user inspects the old roof of their home. The user takes a picture of the roof with their smartphone and sends it to a server for analysis. The server identifies the deteriorated areas, develops a reinforcement plan, and presents it. If the emotion engine detects the user's anxiety, it will show past repair examples from roofing experts to reassure the user.

[0379] An example of a prompt message input to a generated AI model is: "This program combines an AI system that diagnoses the deterioration status of a building with an emotion engine that recognizes user emotions. Based on the information obtained from image data, identify deteriorated areas, develop a reinforcement plan, and provide user support through emotion analysis."

[0380] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0381] Step 1:

[0382] The user uses an image acquisition device to capture image data of the target building or infrastructure. Specifically, they use the camera function of their smartphone to take multiple photos of the area to be diagnosed. The captured image data is saved to the smartphone's memory. The input here is optical information indicating the current state of the building, and the output is data in an image file format (such as JPEG or PNG).

[0383] Step 2:

[0384] The terminal uploads image data acquired from the user's smartphone to the server. In this process, the terminal transmits data to the server using the internet. Specifically, the user selects an image using a dedicated application and presses the send button. The input is image data stored on the terminal, and the output is image data stored on the server's storage.

[0385] Step 3:

[0386] The server analyzes the received image data using a generative AI model. The AI ​​model is trained to identify degradation patterns (cracks, distortion, discoloration, etc.). Specifically, the AI ​​analyzes the pixel information of the image, extracts features, and performs pattern recognition. The input is image data stored on the server, and the output is quantified data indicating the areas of degradation (e.g., crack width, location).

[0387] Step 4:

[0388] The server creates a prioritized reinforcement plan based on the degradation data obtained from the analysis. The server determines the priority of repairs, taking into account the severity and scope of the degradation. Specifically, it analyzes quantitative data and generates repair proposals in accordance with a reinforcement plan template. The input is quantitative data on the degraded areas, and the output is a prioritized reinforcement plan.

[0389] Step 5:

[0390] The server sends the generated augmentation plan to the user via the terminal. The user reviews and understands the augmentation plan on the terminal. Specifically, the user opens the application and views the plan file. The input is the augmentation plan (digital document), and the output is the augmentation plan provided to the user as visual information.

[0391] Step 6:

[0392] The terminal acquires user voice and facial expression data when reviewing the reinforcement plan and sends it to the server. During this process, the terminal uses its camera and microphone to record the user's reactions. The input is the user's voice and video, and the output is emotion data sent to the server.

[0393] Step 7:

[0394] The server uses an emotion engine to analyze the user's voice and facial expressions and perform an emotional assessment. The server recognizes whether the user is experiencing anxiety or concern and generates appropriate additional information. The input is emotional data, and the output is additional information designed to enhance the user's sense of security (e.g., success stories, expert comments).

[0395] Step 8:

[0396] The server provides the user with additional information based on sentiment analysis via the terminal. The user uses this information to deepen their understanding of the reinforcement plan and reduce anxiety. Specifically, the user reviews additional information such as videos and text. The input is supplementary information based on sentiment analysis, and the output is an improvement in the user's psychological sense of security.

[0397] (Application Example 2)

[0398] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0399] In diagnosing deterioration of factory equipment and infrastructure, conventional methods make it difficult to accurately determine the extent of deterioration and formulate reinforcement plans based on the results. Furthermore, there is a lack of care that addresses the anxieties and concerns felt by users. This can lead to delays in timely and appropriate maintenance, potentially reducing work efficiency.

[0400] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0401] In this invention, the server includes means for acquiring image information of an object using an image acquisition device, means for identifying the deterioration status of the object using an automatic processing device that analyzes the image information, means for identifying repair locations and setting their importance based on the identification results, means for creating an optimal reinforcement plan based on the importance, means for presenting the reinforcement plan to the user, and means for detecting the user's emotions and providing information in accordance with those emotions. This makes it possible to accurately diagnose the deterioration of factory equipment and optimize maintenance plans that take into account the user's emotions.

[0402] An "image acquisition device" is a device used to acquire image information of an object.

[0403] An "automated processing device" is a system that analyzes acquired image information to identify the degree of deterioration of an object.

[0404] "Importance" is a measure of priority assigned to the identified repair locations.

[0405] A "reinforcement plan" is a plan for optimized repair and reinforcement based on the identified deterioration status.

[0406] A "user" is a person or organization that operates the system and receives suggestions for reinforcement plans.

[0407] "Providing information tailored to emotions" refers to detecting the user's emotions and then providing additional information to alleviate their anxiety or concerns.

[0408] To implement this invention, first, a camera device capable of photographing equipment and structures within a factory is used as the image acquisition device. The user uses this camera to photograph objects that may be deteriorating. The captured image data is transmitted to the server via a terminal.

[0409] The server uses an automated processing unit, specifically an AI processing module, to analyze the transmitted image data. Through this analysis, the AI ​​system identifies the deterioration patterns of the object and quantifies the degree of deterioration. For this AI processing, for example, open-source image processing libraries such as OpenCV or deep learning frameworks such as TensorFlow can be used.

[0410] Furthermore, the server assigns a priority level to the identified repair locations and creates an optimal reinforcement plan based on this level. This reinforcement plan is created using planning software to develop a feasible maintenance plan tailored to the degree and scale of the identified deterioration.

[0411] Meanwhile, the server uses emotion recognition software to analyze the user's response to the reinforcement plan. This is done by evaluating facial expressions and voice acquired through a webcam and microphone. This emotion evaluation uses an emotion analysis engine such as EmotionEngine. Based on the results, if the user feels anxiety or concern, additional information and specific examples can be provided to support the user according to their needs.

[0412] As a concrete example, consider the case of performing regular inspections of factory equipment. When a user takes photos of the equipment and sends them to the system, the server first uses an AI model to diagnose the degree of deterioration and design a repair plan. At the same time, an emotion engine detects the user's reactions and, if necessary, presents additional information or past success stories, thereby improving the sense of security during the inspection work.

[0413] An example of a prompt to input into the generating AI model is as follows: "Please tell me how to effectively implement a degradation diagnosis support application for use in a factory. In particular, please focus on optimizing information delivery based on the worker's emotions."

[0414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0415] Step 1:

[0416] The user uses a camera device to photograph equipment and structures within the factory and acquires the image data. This image data is transmitted to the server via a terminal. The input is the captured image data, and the output is the transmitted image data.

[0417] Step 2:

[0418] The server uses an AI model to analyze the received image data. Specifically, it uses OpenCV and TensorFlow to perform image processing, identify degradation patterns, and perform classification. The input is the image data sent in step 1, and the output is the identified degradation patterns and classification results.

[0419] Step 3:

[0420] The server sets the importance of the repair locations based on the identified degradation information. Using planning software, it lists the locations in descending order of priority, taking into account the degree of degradation. The input is the identification result from step 2, and the output is a list of repair locations with assigned priorities.

[0421] Step 4:

[0422] The server creates an optimal reinforcement plan based on a list of repair locations. This involves using a planning algorithm to develop specific maintenance procedures and schedules. The input is the priority list from step 3, and the output is a detailed reinforcement plan.

[0423] Step 5:

[0424] The server uses emotion recognition software to analyze the user's facial expressions and voice when presenting reinforcement plans. It uses emotion analysis tools such as EmotionEngine to evaluate emotions like anxiety and concern. The input is user facial expression and voice data, and the output is the result of the emotion evaluation.

[0425] Step 6:

[0426] Based on the emotional assessment results, the server provides the user with additional information and success stories to help alleviate anxiety. This helps the user accept the reinforcement plan with confidence. The input is the emotional assessment result from step 5, and the output is the additional information and success stories presented to the user.

[0427] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0428] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0429] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0430] [Third Embodiment]

[0431] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0432] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0433] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0434] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0435] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0436] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0437] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0438] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0439] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0440] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0441] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0442] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0443] The system of this invention is designed to effectively diagnose the deterioration of buildings and infrastructure and to provide an optimal reinforcement plan. This system mainly consists of an image acquisition device, a server, a terminal, and an interface with the user.

[0444] Users capture images of buildings and infrastructure using image acquisition devices such as smartphones. This image data includes the exterior and interior conditions of buildings and contains important information. The device transmits this image data to a server for further analysis.

[0445] The server analyzes the received image data using AI technology. Specifically, the AI ​​model automatically analyzes the images and identifies specific degradation patterns. In this process, the AI ​​identifies specific anomalies such as cracks, corrosion, and rust, and quantifies the degree of degradation.

[0446] After the analysis to assess the degree of deterioration is complete, the server identifies areas that require repair. Furthermore, the AI ​​automatically determines the repair priority based on the severity and scope of the deterioration, as well as the user's budget and priorities.

[0447] Based on these priorities, the server creates an optimal reinforcement plan for the user. This plan includes the necessary steps, materials, and schedule for repairing deterioration, and is presented in a format that is easy for the user to understand.

[0448] As a concrete example, suppose a local government uses this system to structurally evaluate aging bridges within its area. The user, a local government employee, photographs various parts of the bridge and sends the images to the server. The server uses AI to analyze the images, detecting deterioration of support pillars and cable wear. Based on the results, areas requiring urgent repair are identified, and an optimal repair schedule and budget proposal are provided to the local government. This process not only significantly improves the accuracy and efficiency of repairs but also helps to effectively utilize limited resources.

[0449] The following describes the processing flow.

[0450] Step 1:

[0451] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure to be diagnosed. Once the capture is complete, the device prepares the image data and gets ready to send it to the server.

[0452] Step 2:

[0453] The device compresses the image data captured by the user, adds metadata (e.g., date and time of capture and location information), and sends it to the server via a secure communication protocol.

[0454] Step 3:

[0455] The server temporarily stores the received image data in storage and converts the images into a format that can be used for AI analysis. Specifically, it adjusts the image size and removes noise, preparing the images for feature extraction.

[0456] Step 4:

[0457] The server inputs the formatted image data into an AI model to analyze the deterioration of buildings and infrastructure. The AI ​​model identifies and quantitatively evaluates anomalies such as cracks, corrosion, and rust. Based on this output, the degree and extent of deterioration are determined.

[0458] Step 5:

[0459] The server identifies areas requiring repair based on analysis results obtained by AI. Furthermore, it automatically determines the priority of repair work by considering multiple factors (severity of deterioration, budget, priorities, etc.).

[0460] Step 6:

[0461] The server creates an optimal reinforcement plan based on prioritized information. This plan is designed to include specific repair steps, materials to be used, and a schedule.

[0462] Step 7:

[0463] The terminal presents the user with an augmentation plan created by the server. The user can review the augmentation plan and send feedback to the server as needed.

[0464] Step 8:

[0465] The server analyzes user feedback and additional information, updating reinforcement plans as needed. Ultimately, it can be linked with a regional information database to store repair data and use it for disaster response.

[0466] (Example 1)

[0467] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] There is a need to diagnose the deterioration of buildings and infrastructure early and accurately, and to provide efficient and optimal reinforcement plans. However, conventional methods have limitations in the accuracy and efficiency of identifying deterioration and formulating repair plans, and there are challenges in appropriately allocating resources.

[0469] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0470] In this invention, the server includes means for collecting image information of a target facility using an image acquisition function, means for determining the extent of damage to the facility using artificial intelligence that analyzes the image information, and means for identifying repair locations and setting their importance based on the determination results. This makes it possible to provide efficient and highly accurate deterioration diagnosis and repair plans.

[0471] The "image acquisition function" is a function used to capture the exterior and interior condition of the target facility, and primarily collects information using image sensors.

[0472] "Image information" refers to digital information that indicates the condition of the target facility, and this includes data acquired as photographs and videos.

[0473] "Artificial intelligence" is a technology that analyzes large amounts of data to identify specific patterns or anomalies, and in this invention, it is used in particular to determine the state of deterioration.

[0474] "Means for determining the extent of damage" refers to methods for analyzing acquired image information to evaluate the condition of the target facility and identify damaged or deteriorated areas.

[0475] "Methods for identifying repair locations" refer to methods that, based on the results of artificial intelligence analysis, clearly indicate the specific areas that require repair.

[0476] A "means for setting importance" refers to a method for automatically determining the priority of repairs, taking into account the severity and scope of the damage.

[0477] A "repair plan" is a plan that includes specific repair methods, resource allocation, and schedules for identified repair areas.

[0478] A "generative AI model" is an artificial intelligence model that generates information through natural language processing and image analysis to improve the accuracy of planning.

[0479] A "prompt statement" is an instruction statement used to convey requests to a generating AI model, and is used when a user customizes a repair plan.

[0480] This invention specifically describes a system for diagnosing the deterioration of buildings and infrastructure and providing appropriate repair plans. The system mainly consists of an image acquisition function, a server, a terminal, and a user interface.

[0481] Users collect image information of target structures using image acquisition devices such as smartphones and drones. These devices are equipped with high-precision cameras, enabling the collection of multifaceted and detailed data. This makes it possible to capture even small damages and deterioration that are not visible to the naked eye.

[0482] The terminal is responsible for transmitting the collected image information to the server. The data is properly compressed and transferred to the server quickly and securely. Compression techniques are used that do not compromise image quality and maintain data accuracy.

[0483] When the server receives image information, it performs analysis using artificial intelligence. Specifically, a generative AI model analyzes the image using a deep learning algorithm to automatically identify deterioration patterns such as cracks and corrosion. The server extracts features from the image and determines the extent of the damage by comparing them with an existing database.

[0484] Once the analysis is complete, the server identifies the repair locations based on the results and sets their importance level. The importance level is determined based on the severity and scope of the damage, as well as the budget and priorities provided by the user. During this process, users can input additional information and requests using prompts. For example, by telling the server, "This part is particularly important; please prioritize its repair schedule," a more precise repair plan can be generated.

[0485] As a concrete example, consider a case where a city hall employee inspects a local bridge. The employee takes detailed photographs of the aging parts of the bridge and sends the images to a server via a terminal. The server immediately analyzes the images and identifies deterioration of the bridge's supports and cables. Based on the analysis results, it identifies the most urgent repair areas and sets repair priorities. This information is provided to the employee in a user-friendly format, enabling quick and accurate decision-making.

[0486] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0487] Step 1:

[0488] The user uses an image acquisition device to capture images of the target structure. The captured images contain detailed information about the structure from multiple angles. The input is the physical structure, and the output is digital image data. The user selects the necessary images and performs the capture operation.

[0489] Step 2:

[0490] The terminal compresses and encrypts the collected image data before sending it to the server. This ensures the efficiency and security of the transfer. The input is the captured image data, and the output is the compressed and encrypted data. The terminal then performs the operation of transferring this data to the server at high speed.

[0491] Step 3:

[0492] The server decompresses the received compressed image data and prepares it for analysis. The input is encrypted compressed data, and the output is image data in a format suitable for analysis. After decompression, the server organizes the data for artificial intelligence analysis.

[0493] Step 4:

[0494] The server automatically identifies degradation patterns from image data using a generative AI model. This process employs deep learning algorithms to extract features from the images. The input is the decompressed image data, and the output is the identified degradation patterns and evaluation results. The server then analyzes this data and performs an evaluation of the structure's condition.

[0495] Step 5:

[0496] The server identifies and prioritizes repair locations based on the degradation assessment results. The AI ​​lists the necessary repair locations and determines their priority based on the severity and extent of the damage. The input is the degradation pattern assessment results, and the output is a list of prioritized repair locations. The server analyzes the data and selects the most important repair targets.

[0497] Step 6:

[0498] The server receives prompts from the user and makes adjustments to reflect them in the repair plan. Prompts allow the user to provide additional instructions or requests. The input is the user's prompts, and the output is the adjusted repair plan. The server considers the user's needs and flexibly modifies the plan.

[0499] Step 7:

[0500] The server creates and provides the user with a final repair plan. This plan includes specific repair steps, a list of necessary materials, and a schedule. The input is coordinated repair data and priority information, and the output is a user-friendly plan. The server organizes the data, constructs the plan, and presents it to the user.

[0501] (Application Example 1)

[0502] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0503] The deterioration of buildings and infrastructure, leading to reduced safety and increased repair costs due to the progression of deterioration, are significant challenges in the management of public and large-scale facilities. Conventional methods struggle to accurately assess the extent of deterioration and formulate efficient reinforcement plans. Especially in large-scale facilities, constant monitoring is required, necessitating real-time information gathering and feedback of the results.

[0504] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0505] In this invention, the server includes means for acquiring image data of a target facility using an image acquisition device, means for determining the deterioration status of the facility using artificial intelligence that analyzes the image data, and means for notifying abnormalities in real time using monitoring equipment. This enables real-time monitoring of facility deterioration, allowing for rapid response and the formulation of efficient reinforcement plans.

[0506] An "image acquisition device" is a device used to acquire image data of a target facility, and includes photographic equipment such as cameras.

[0507] "Target facilities" is a general term for buildings and infrastructure that are subject to deterioration assessment.

[0508] "Image data" refers to digital data containing visual information acquired by an image acquisition device.

[0509] "Artificial intelligence" refers to a computer program or algorithm used to analyze image data and determine its degree of deterioration.

[0510] "Discrimination results" refer to data indicating the judgment regarding the degree of deterioration obtained through analysis by artificial intelligence.

[0511] "Areas requiring repair" refers to parts or areas of a facility that require restoration or reinforcement, as identified based on the assessment results.

[0512] "Priority" refers to a ranking assigned to determine the order in which work is carried out when repairing or reinforcing areas.

[0513] A "reinforcement plan" is a plan that is constructed based on the degree of deterioration and includes the repair methods, materials to be used, and schedule for the areas that need repair.

[0514] "Users" refers to those who receive a presentation of the reinforcement plan, including facility managers and owners.

[0515] "Monitoring equipment" refers to devices used to patrol the interior of a facility and diagnose deterioration in real time, and typically includes mobile robots.

[0516] "Communication equipment" refers to devices used by users to send and receive data, and includes smartphones and computers.

[0517] The system for implementing this invention is configured as follows.

[0518] First, the user uses an image acquisition device to capture images of various parts of the target facility. High-resolution cameras mounted on smartphones or monitoring equipment are used as image acquisition devices, allowing for efficient capture of deteriorated areas of the target facility. The captured image data is then transmitted to a server via a communication device.

[0519] The server analyzes the received image data using artificial intelligence. This analysis utilizes deep learning frameworks such as TensorFlow and PyTorch, and the generative AI model automatically identifies degradation patterns. Specifically, the data processing involves pre-processing the images to identify specific degradation forms such as cracks and corrosion, and then scoring their severity.

[0520] Next, the server identifies areas requiring repair based on the assessment results and assigns a priority to each area. Prioritization is determined based on the severity and scope of deterioration, as well as the user's budget and perceived importance. Based on this, an optimal reinforcement plan is created and presented to the user as documentation. This reinforcement plan includes the necessary processes, materials to be used, and construction schedule. Users can input feedback and additional requests regarding this plan using a communication device. This information is also processed by the server and incorporated into the plan.

[0521] As a concrete example, if a monitoring device detects a crack in a wall during a patrol at a commercial facility, the data is immediately sent to a server for analysis, after which repair work for the following day is planned. An example of a prompt message to the generated AI model would be, "Diagnose the deterioration status of the facility from the images captured by the camera and generate the necessary repair plan." The introduction of this system will improve the safety and efficiency of facility management.

[0522] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0523] Step 1:

[0524] The user uses an image acquisition device to acquire images of the target facility. The input is images of the facility's exterior and specific interior areas, generating high-resolution image data. The output is this acquired image data.

[0525] Step 2:

[0526] The user uses a communication device to send acquired image data to the server. The input is high-resolution image data, and the output is image data securely transmitted to the server. In this step, data transfer is performed using a communication protocol.

[0527] Step 3:

[0528] The server inputs the received image data into the AI ​​analysis system. The input is the transmitted image data, and the output is the analysis result of the degradation patterns contained in the image. As part of the data processing, image preprocessing is performed, and patterns such as cracks and corrosion are identified using a generated AI model.

[0529] Step 4:

[0530] Based on the analysis results, the server uses AI to identify areas requiring repair and their priority. The input is analyzed degradation data, and the output is a list of degraded areas and their priorities. A ranking is generated using a prioritization algorithm based on the severity of the degradation.

[0531] Step 5:

[0532] The server generates a reinforcement plan based on identified repair locations and their priorities. The input is a list of prioritized repair locations, and the output is a detailed reinforcement plan. The plan includes materials to be used, work processes, and schedules, and an optimization algorithm is employed.

[0533] Step 6:

[0534] The server presents the generated augmentation plan to the user. The input is the created augmentation plan, and the output is information about the augmentation plan displayed in an easy-to-understand format. Details of the plan are visualized through the user interface.

[0535] Step 7:

[0536] Users input additional information regarding reinforcement plans via a communication device and send it to the server. The input consists of supplementary information and requested modifications, while the output is updated plan information. This allows for flexible plan adjustments.

[0537] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0538] This invention combines a system that diagnoses the deterioration status of buildings and infrastructure and provides an optimal reinforcement plan with an emotion engine that recognizes the user's emotions. This system mainly consists of an image acquisition device, a server, a terminal, and an emotion engine.

[0539] The user takes a picture of the target structure using an image acquisition device such as a smartphone and sends the image data to the terminal. The terminal sends the image data to a server, which analyzes the received image using an AI model. The AI ​​identifies specific deterioration patterns from the image and quantifies the degree of deterioration.

[0540] Next, the server identifies areas requiring repair based on its degradation status and prioritizes repairs by considering multiple factors. Based on this, an optimal reinforcement plan is formulated, and the emotion engine further analyzes user reactions.

[0541] The emotion engine analyzes the user's voice and facial expressions as they review the reinforcement plan, and evaluates their emotions. This information is then used to suggest additional information and support that alleviate the user's anxiety and concerns. For example, if the emotion engine senses that the user is anxious, it can provide detailed explanations and present actual repair examples to address that anxiety.

[0542] As a concrete example, suppose an individual user uses this system to inspect the old roof of their home. The user takes photos of the roof and sends them to the server, where the AI ​​identifies deteriorated areas. The server evaluates the extent of the deterioration and creates a reinforcement plan for areas that need repair. If the user feels uneasy after reviewing the plan, the emotion engine detects this and presents successful repair case studies by roofing professionals to provide reassurance. In this way, user satisfaction can be increased by providing an integrated solution that combines technical problem diagnosis with emotional support.

[0543] The following describes the processing flow.

[0544] Step 1:

[0545] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure they want to diagnose. After taking the pictures, the image data is saved to the device.

[0546] Step 2:

[0547] The terminal compresses the stored image data, adds the necessary metadata, and sends it to the server. A secure communication protocol is used for transmission to avoid data loss.

[0548] Step 3:

[0549] The server preprocesses the received image data before inputting it into the analysis system. This preprocessing includes resizing and noise reduction, preparing the images for analysis.

[0550] Step 4:

[0551] The server inputs pre-processed image data into an AI model to analyze the degradation status. The AI ​​model identifies anomalies in the image (e.g., cracks and rust), quantifies them, and generates an evaluation result.

[0552] Step 5:

[0553] The server identifies areas requiring repair based on evaluation results obtained from the AI ​​model. Next, it considers multiple factors such as the degree of deterioration, the importance of the structure, and the user's budget to determine the priority of repairs.

[0554] Step 6:

[0555] The server creates a detailed reinforcement plan based on the prioritized results. This plan includes repair processes, materials to be used, and implementation schedules, and is compiled in an easily viewable format such as PDF or images.

[0556] Step 7:

[0557] When the terminal presents the augmentation plan received from the server to the user, the emotion engine is also activated. When the user reviews the augmentation plan, the emotion engine performs voice analysis and facial recognition to detect the user's emotions.

[0558] Step 8:

[0559] Based on the user's emotional information obtained by the emotion engine, the server provides additional information to alleviate the user's anxiety and concerns about the plan. For example, if anxiety is detected, it may present past success stories or provide more detailed explanations.

[0560] Step 9:

[0561] After receiving additional information and support, users will be able to enter questions and additional instructions regarding the reinforcement plan into the terminal. The terminal will collect this information and send it to the server in real time.

[0562] Step 10:

[0563] The server analyzes additional information received from the user and updates the reinforcement plan if necessary. The updated plan is then sent to the terminal and presented to the user, optimizing the plan.

[0564] (Example 2)

[0565] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0566] Conventional building and infrastructure deterioration assessment systems focused on technical assessments and the presentation of reinforcement plans, but lacked sufficient support that considered the user's psychological state. As a result, user anxieties and concerns about the proposed reinforcement plans were not addressed, leading to low user satisfaction with the assessment results and proposals. Furthermore, efficient information management that takes into account region-specific conditions remained a challenge.

[0567] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0568] In this invention, the server includes means for determining the deterioration status using artificial intelligence to analyze image data, means for creating and presenting reinforcement plans, emotion engine means for detecting user emotions and providing appropriate additional information, and means for analyzing user emotion data and proposing psychological support. This makes it possible to not only provide technical diagnostic results but also to provide comprehensive support that takes user emotions into consideration, thereby improving user satisfaction with deterioration diagnosis and reinforcement proposals.

[0569] An "image acquisition device" refers to equipment used to acquire image data of structures such as buildings and infrastructure. Specifically, this includes smartphones and digital cameras equipped with camera functions.

[0570] "Artificial intelligence" is a technology developed to allow computer systems to mimic human intelligence, and is primarily used for data analysis and pattern recognition.

[0571] "Deterioration status" refers to the condition or degree to which a structure has deteriorated due to aging or environmental factors, and whether repairs are necessary.

[0572] A "reinforcement plan" is a plan that outlines specific procedures and methods for repairing or improving deteriorated structures, and also includes their priorities.

[0573] An "emotion engine" refers to a technology or system that analyzes a user's voice and facial expressions to recognize their emotions and provide appropriate support or additional information.

[0574] "Prioritizing" means evaluating the importance and necessity of multiple options or tasks, and deciding the order in which to perform them.

[0575] "Psychological support" refers to activities and information provided that take emotional aspects into consideration in order to alleviate users' anxiety and concerns and provide them with a sense of security.

[0576] "Local information" refers to geographical, climatic, social, and economic data and knowledge related to a specific region, and is used for disaster response and planning.

[0577] This system consists of an image acquisition device, a server, a terminal, and an emotion engine. First, the user acquires image data of buildings and infrastructure using an image acquisition device such as a smartphone or digital camera. The image data captured by the user is sent to a terminal, such as a smartphone or personal computer. The terminal then sends this image data to the server.

[0578] The server possesses powerful computing capabilities and analyzes the received image data based on a generative AI model. This generative AI model is built using deep learning techniques to identify deterioration patterns from building images. Typical frameworks used include TensorFlow and PyTorch. The server detects signs of deterioration in the images and quantifies the degree of deterioration.

[0579] Furthermore, based on the analysis results, the server identifies where repairs are needed, prioritizes them, and creates an optimal reinforcement plan. This plan is presented to the user via their terminal. For example, it utilizes data on the extent and location of cracks in the building to specifically show effective reinforcement methods and procedures.

[0580] Subsequently, the user reviews the reinforcement plan using a device. During this process, the device captures the user's facial expressions and voice data, which are then analyzed by a server using an emotion engine. This emotion engine has the technology to identify whether the user has any anxieties or doubts about the plan. Based on the detected emotions, additional information is provided to alleviate the user's anxiety. For example, actual examples of successful reinforcements and testimonials from construction companies may be presented.

[0581] As a concrete example, consider a case where a user inspects the old roof of their home. The user takes a picture of the roof with their smartphone and sends it to a server for analysis. The server identifies the deteriorated areas, develops a reinforcement plan, and presents it. If the emotion engine detects the user's anxiety, it will show past repair examples from roofing experts to reassure the user.

[0582] An example of a prompt message input to a generated AI model is: "This program combines an AI system that diagnoses the deterioration status of a building with an emotion engine that recognizes user emotions. Based on the information obtained from image data, identify deteriorated areas, develop a reinforcement plan, and provide user support through emotion analysis."

[0583] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0584] Step 1:

[0585] The user uses an image acquisition device to capture image data of the target building or infrastructure. Specifically, they use the camera function of their smartphone to take multiple photos of the area to be diagnosed. The captured image data is saved to the smartphone's memory. The input here is optical information indicating the current state of the building, and the output is data in an image file format (such as JPEG or PNG).

[0586] Step 2:

[0587] The terminal uploads image data acquired from the user's smartphone to the server. In this process, the terminal transmits data to the server using the internet. Specifically, the user selects an image using a dedicated application and presses the send button. The input is image data stored on the terminal, and the output is image data stored on the server's storage.

[0588] Step 3:

[0589] The server analyzes the received image data using a generative AI model. The AI ​​model is trained to identify degradation patterns (cracks, distortion, discoloration, etc.). Specifically, the AI ​​analyzes the pixel information of the image, extracts features, and performs pattern recognition. The input is image data stored on the server, and the output is quantified data indicating the areas of degradation (e.g., crack width, location).

[0590] Step 4:

[0591] The server creates a prioritized reinforcement plan based on the degradation data obtained from the analysis. The server determines the priority of repairs, taking into account the severity and scope of the degradation. Specifically, it analyzes quantitative data and generates repair proposals in accordance with a reinforcement plan template. The input is quantitative data on the degraded areas, and the output is a prioritized reinforcement plan.

[0592] Step 5:

[0593] The server sends the generated augmentation plan to the user via the terminal. The user reviews and understands the augmentation plan on the terminal. Specifically, the user opens the application and views the plan file. The input is the augmentation plan (digital document), and the output is the augmentation plan provided to the user as visual information.

[0594] Step 6:

[0595] The terminal acquires user voice and facial expression data when reviewing the reinforcement plan and sends it to the server. During this process, the terminal uses its camera and microphone to record the user's reactions. The input is the user's voice and video, and the output is emotion data sent to the server.

[0596] Step 7:

[0597] The server uses an emotion engine to analyze the user's voice and facial expressions and perform an emotional assessment. The server recognizes whether the user is experiencing anxiety or concern and generates appropriate additional information. The input is emotional data, and the output is additional information designed to enhance the user's sense of security (e.g., success stories, expert comments).

[0598] Step 8:

[0599] The server provides the user with additional information based on sentiment analysis via the terminal. The user uses this information to deepen their understanding of the reinforcement plan and reduce anxiety. Specifically, the user reviews additional information such as videos and text. The input is supplementary information based on sentiment analysis, and the output is an improvement in the user's psychological sense of security.

[0600] (Application Example 2)

[0601] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0602] In diagnosing deterioration of factory equipment and infrastructure, conventional methods make it difficult to accurately determine the extent of deterioration and formulate reinforcement plans based on the results. Furthermore, there is a lack of care that addresses the anxieties and concerns felt by users. This can lead to delays in timely and appropriate maintenance, potentially reducing work efficiency.

[0603] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0604] In this invention, the server includes means for acquiring image information of an object using an image acquisition device, means for identifying the deterioration status of the object using an automatic processing device that analyzes the image information, means for identifying repair locations and setting their importance based on the identification results, means for creating an optimal reinforcement plan based on the importance, means for presenting the reinforcement plan to the user, and means for detecting the user's emotions and providing information in accordance with those emotions. This makes it possible to accurately diagnose the deterioration of factory equipment and optimize maintenance plans that take into account the user's emotions.

[0605] An "image acquisition device" is a device used to acquire image information of an object.

[0606] An "automated processing device" is a system that analyzes acquired image information to identify the degree of deterioration of an object.

[0607] "Importance" is a measure of priority assigned to the identified repair locations.

[0608] A "reinforcement plan" is a plan for optimized repair and reinforcement based on the identified deterioration status.

[0609] A "user" is a person or organization that operates the system and receives suggestions for reinforcement plans.

[0610] "Providing information tailored to emotions" refers to detecting the user's emotions and then providing additional information to alleviate their anxiety or concerns.

[0611] To implement this invention, first, a camera device capable of photographing equipment and structures within a factory is used as the image acquisition device. The user uses this camera to photograph objects that may be deteriorating. The captured image data is transmitted to the server via a terminal.

[0612] The server uses an automated processing unit, specifically an AI processing module, to analyze the transmitted image data. Through this analysis, the AI ​​system identifies the deterioration patterns of the object and quantifies the degree of deterioration. For this AI processing, for example, open-source image processing libraries such as OpenCV or deep learning frameworks such as TensorFlow can be used.

[0613] Furthermore, the server assigns a priority level to the identified repair locations and creates an optimal reinforcement plan based on this level. This reinforcement plan is created using planning software to develop a feasible maintenance plan tailored to the degree and scale of the identified deterioration.

[0614] Meanwhile, the server uses emotion recognition software to analyze the user's response to the reinforcement plan. This is done by evaluating facial expressions and voice acquired through a webcam and microphone. This emotion evaluation uses an emotion analysis engine such as EmotionEngine. Based on the results, if the user feels anxiety or concern, additional information and specific examples can be provided to support the user according to their needs.

[0615] As a concrete example, consider the case of performing regular inspections of factory equipment. When a user takes photos of the equipment and sends them to the system, the server first uses an AI model to diagnose the degree of deterioration and design a repair plan. At the same time, an emotion engine detects the user's reactions and, if necessary, presents additional information or past success stories, thereby improving the sense of security during the inspection work.

[0616] An example of a prompt to input into the generating AI model is as follows: "Please tell me how to effectively implement a degradation diagnosis support application for use in a factory. In particular, please focus on optimizing information delivery based on the worker's emotions."

[0617] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0618] Step 1:

[0619] The user uses a camera device to photograph equipment and structures within the factory and acquires the image data. This image data is transmitted to the server via a terminal. The input is the captured image data, and the output is the transmitted image data.

[0620] Step 2:

[0621] The server uses an AI model to analyze the received image data. Specifically, it uses OpenCV and TensorFlow to perform image processing, identify degradation patterns, and perform classification. The input is the image data sent in step 1, and the output is the identified degradation patterns and classification results.

[0622] Step 3:

[0623] The server sets the importance of the repair locations based on the identified degradation information. Using planning software, it lists the locations in descending order of priority, taking into account the degree of degradation. The input is the identification result from step 2, and the output is a list of repair locations with assigned priorities.

[0624] Step 4:

[0625] The server creates an optimal reinforcement plan based on a list of repair locations. This involves using a planning algorithm to develop specific maintenance procedures and schedules. The input is the priority list from step 3, and the output is a detailed reinforcement plan.

[0626] Step 5:

[0627] The server uses emotion recognition software to analyze the user's facial expressions and voice when presenting reinforcement plans. It uses emotion analysis tools such as EmotionEngine to evaluate emotions like anxiety and concern. The input is user facial expression and voice data, and the output is the result of the emotion evaluation.

[0628] Step 6:

[0629] Based on the emotional assessment results, the server provides the user with additional information and success stories to help alleviate anxiety. This helps the user accept the reinforcement plan with confidence. The input is the emotional assessment result from step 5, and the output is the additional information and success stories presented to the user.

[0630] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0631] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0632] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0633] [Fourth Embodiment]

[0634] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0635] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0636] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0637] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0638] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0639] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0640] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0641] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0642] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0643] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0644] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0645] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0646] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] The system of this invention is designed to effectively diagnose the deterioration of buildings and infrastructure and to provide an optimal reinforcement plan. This system mainly consists of an image acquisition device, a server, a terminal, and an interface with the user.

[0648] Users capture images of buildings and infrastructure using image acquisition devices such as smartphones. This image data includes the exterior and interior conditions of buildings and contains important information. The device transmits this image data to a server for further analysis.

[0649] The server analyzes the received image data using AI technology. Specifically, the AI ​​model automatically analyzes the images and identifies specific degradation patterns. In this process, the AI ​​identifies specific anomalies such as cracks, corrosion, and rust, and quantifies the degree of degradation.

[0650] After the analysis to assess the degree of deterioration is complete, the server identifies areas that require repair. Furthermore, the AI ​​automatically determines the repair priority based on the severity and scope of the deterioration, as well as the user's budget and priorities.

[0651] Based on these priorities, the server creates an optimal reinforcement plan for the user. This plan includes the necessary steps, materials, and schedule for repairing deterioration, and is presented in a format that is easy for the user to understand.

[0652] As a concrete example, suppose a local government uses this system to structurally evaluate aging bridges within its area. The user, a local government employee, photographs various parts of the bridge and sends the images to the server. The server uses AI to analyze the images, detecting deterioration of support pillars and cable wear. Based on the results, areas requiring urgent repair are identified, and an optimal repair schedule and budget proposal are provided to the local government. This process not only significantly improves the accuracy and efficiency of repairs but also helps to effectively utilize limited resources.

[0653] The following describes the processing flow.

[0654] Step 1:

[0655] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure to be diagnosed. Once the capture is complete, the device prepares the image data and gets ready to send it to the server.

[0656] Step 2:

[0657] The device compresses the image data captured by the user, adds metadata (e.g., date and time of capture and location information), and sends it to the server via a secure communication protocol.

[0658] Step 3:

[0659] The server temporarily stores the received image data in storage and converts the images into a format that can be used for AI analysis. Specifically, it adjusts the image size and removes noise, preparing the images for feature extraction.

[0660] Step 4:

[0661] The server inputs the formatted image data into an AI model to analyze the deterioration of buildings and infrastructure. The AI ​​model identifies and quantitatively evaluates anomalies such as cracks, corrosion, and rust. Based on this output, the degree and extent of deterioration are determined.

[0662] Step 5:

[0663] The server identifies areas requiring repair based on analysis results obtained by AI. Furthermore, it automatically determines the priority of repair work by considering multiple factors (severity of deterioration, budget, priorities, etc.).

[0664] Step 6:

[0665] The server creates an optimal reinforcement plan based on prioritized information. This plan is designed to include specific repair steps, materials to be used, and a schedule.

[0666] Step 7:

[0667] The terminal presents the user with an augmentation plan created by the server. The user can review the augmentation plan and send feedback to the server as needed.

[0668] Step 8:

[0669] The server analyzes user feedback and additional information, updating reinforcement plans as needed. Ultimately, it can be linked with a regional information database to store repair data and use it for disaster response.

[0670] (Example 1)

[0671] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0672] There is a need to diagnose the deterioration of buildings and infrastructure early and accurately, and to provide efficient and optimal reinforcement plans. However, conventional methods have limitations in the accuracy and efficiency of identifying deterioration and formulating repair plans, and there are challenges in appropriately allocating resources.

[0673] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0674] In this invention, the server includes means for collecting image information of a target facility using an image acquisition function, means for determining the extent of damage to the facility using artificial intelligence that analyzes the image information, and means for identifying repair locations and setting their importance based on the determination results. This makes it possible to provide efficient and highly accurate deterioration diagnosis and repair plans.

[0675] The "image acquisition function" is a function used to capture the exterior and interior condition of the target facility, and primarily collects information using image sensors.

[0676] "Image information" refers to digital information that indicates the condition of the target facility, and this includes data acquired as photographs and videos.

[0677] "Artificial intelligence" is a technology that analyzes large amounts of data to identify specific patterns or anomalies, and in this invention, it is used in particular to determine the state of deterioration.

[0678] "Means for determining the extent of damage" refers to methods for analyzing acquired image information to evaluate the condition of the target facility and identify damaged or deteriorated areas.

[0679] "Methods for identifying repair locations" refer to methods that, based on the results of artificial intelligence analysis, clearly indicate the specific areas that require repair.

[0680] A "means for setting importance" refers to a method for automatically determining the priority of repairs, taking into account the severity and scope of the damage.

[0681] A "repair plan" is a plan that includes specific repair methods, resource allocation, and schedules for identified repair areas.

[0682] A "generative AI model" is an artificial intelligence model that generates information through natural language processing and image analysis to improve the accuracy of planning.

[0683] A "prompt statement" is an instruction statement used to convey requests to a generating AI model, and is used when a user customizes a repair plan.

[0684] This invention specifically describes a system for diagnosing the deterioration of buildings and infrastructure and providing appropriate repair plans. The system mainly consists of an image acquisition function, a server, a terminal, and a user interface.

[0685] Users collect image information of target structures using image acquisition devices such as smartphones and drones. These devices are equipped with high-precision cameras, enabling the collection of multifaceted and detailed data. This makes it possible to capture even small damages and deterioration that are not visible to the naked eye.

[0686] The terminal is responsible for transmitting the collected image information to the server. The data is properly compressed and transferred to the server quickly and securely. Compression techniques are used that do not compromise image quality and maintain data accuracy.

[0687] When the server receives image information, it performs analysis using artificial intelligence. Specifically, a generative AI model analyzes the image using a deep learning algorithm to automatically identify deterioration patterns such as cracks and corrosion. The server extracts features from the image and determines the extent of the damage by comparing them with an existing database.

[0688] Once the analysis is complete, the server identifies the repair locations based on the results and sets their importance level. The importance level is determined based on the severity and scope of the damage, as well as the budget and priorities provided by the user. During this process, users can input additional information and requests using prompts. For example, by telling the server, "This part is particularly important; please prioritize its repair schedule," a more precise repair plan can be generated.

[0689] As a concrete example, consider a case where a city hall employee inspects a local bridge. The employee takes detailed photographs of the aging parts of the bridge and sends the images to a server via a terminal. The server immediately analyzes the images and identifies deterioration of the bridge's supports and cables. Based on the analysis results, it identifies the most urgent repair areas and sets repair priorities. This information is provided to the employee in a user-friendly format, enabling quick and accurate decision-making.

[0690] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0691] Step 1:

[0692] The user uses an image acquisition device to capture images of the target structure. The captured images contain detailed information about the structure from multiple angles. The input is the physical structure, and the output is digital image data. The user selects the necessary images and performs the capture operation.

[0693] Step 2:

[0694] The terminal compresses and encrypts the collected image data before sending it to the server. This ensures the efficiency and security of the transfer. The input is the captured image data, and the output is the compressed and encrypted data. The terminal then performs the operation of transferring this data to the server at high speed.

[0695] Step 3:

[0696] The server decompresses the received compressed image data and prepares it for analysis. The input is encrypted compressed data, and the output is image data in a format suitable for analysis. After decompression, the server organizes the data for artificial intelligence analysis.

[0697] Step 4:

[0698] The server automatically identifies degradation patterns from image data using a generative AI model. This process employs deep learning algorithms to extract features from the images. The input is the decompressed image data, and the output is the identified degradation patterns and evaluation results. The server then analyzes this data and performs an evaluation of the structure's condition.

[0699] Step 5:

[0700] The server identifies and prioritizes repair locations based on the degradation assessment results. The AI ​​lists the necessary repair locations and determines their priority based on the severity and extent of the damage. The input is the degradation pattern assessment results, and the output is a list of prioritized repair locations. The server analyzes the data and selects the most important repair targets.

[0701] Step 6:

[0702] The server receives prompts from the user and makes adjustments to reflect them in the repair plan. Prompts allow the user to provide additional instructions or requests. The input is the user's prompts, and the output is the adjusted repair plan. The server considers the user's needs and flexibly modifies the plan.

[0703] Step 7:

[0704] The server creates and provides the user with a final repair plan. This plan includes specific repair steps, a list of necessary materials, and a schedule. The input is coordinated repair data and priority information, and the output is a user-friendly plan. The server organizes the data, constructs the plan, and presents it to the user.

[0705] (Application Example 1)

[0706] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0707] The deterioration of buildings and infrastructure, leading to reduced safety and increased repair costs due to the progression of deterioration, are significant challenges in the management of public and large-scale facilities. Conventional methods struggle to accurately assess the extent of deterioration and formulate efficient reinforcement plans. Especially in large-scale facilities, constant monitoring is required, necessitating real-time information gathering and feedback of the results.

[0708] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0709] In this invention, the server includes means for acquiring image data of a target facility using an image acquisition device, means for determining the deterioration status of the facility using artificial intelligence that analyzes the image data, and means for notifying abnormalities in real time using monitoring equipment. This enables real-time monitoring of facility deterioration, allowing for rapid response and the formulation of efficient reinforcement plans.

[0710] An "image acquisition device" is a device used to acquire image data of a target facility, and includes photographic equipment such as cameras.

[0711] "Target facilities" is a general term for buildings and infrastructure that are subject to deterioration assessment.

[0712] "Image data" refers to digital data containing visual information acquired by an image acquisition device.

[0713] "Artificial intelligence" refers to a computer program or algorithm used to analyze image data and determine its degree of deterioration.

[0714] "Discrimination results" refer to data indicating the judgment regarding the degree of deterioration obtained through analysis by artificial intelligence.

[0715] "Areas requiring repair" refers to parts or areas of a facility that require restoration or reinforcement, as identified based on the assessment results.

[0716] "Priority" refers to a ranking assigned to determine the order in which work is carried out when repairing or reinforcing areas.

[0717] A "reinforcement plan" is a plan that is constructed based on the degree of deterioration and includes the repair methods, materials to be used, and schedule for the areas that need repair.

[0718] "Users" refers to those who receive a presentation of the reinforcement plan, including facility managers and owners.

[0719] "Monitoring equipment" refers to devices used to patrol the interior of a facility and diagnose deterioration in real time, and typically includes mobile robots.

[0720] "Communication equipment" refers to devices used by users to send and receive data, and includes smartphones and computers.

[0721] The system for implementing this invention is configured as follows.

[0722] First, the user uses an image acquisition device to capture images of various parts of the target facility. High-resolution cameras mounted on smartphones or monitoring equipment are used as image acquisition devices, allowing for efficient capture of deteriorated areas of the target facility. The captured image data is then transmitted to a server via a communication device.

[0723] The server analyzes the received image data using artificial intelligence. This analysis utilizes deep learning frameworks such as TensorFlow and PyTorch, and the generative AI model automatically identifies degradation patterns. Specifically, the data processing involves pre-processing the images to identify specific degradation forms such as cracks and corrosion, and then scoring their severity.

[0724] Next, the server identifies areas requiring repair based on the assessment results and assigns a priority to each area. Prioritization is determined based on the severity and scope of deterioration, as well as the user's budget and perceived importance. Based on this, an optimal reinforcement plan is created and presented to the user as documentation. This reinforcement plan includes the necessary processes, materials to be used, and construction schedule. Users can input feedback and additional requests regarding this plan using a communication device. This information is also processed by the server and incorporated into the plan.

[0725] As a concrete example, if a monitoring device detects a crack in a wall during a patrol at a commercial facility, the data is immediately sent to a server for analysis, after which repair work for the following day is planned. An example of a prompt message to the generated AI model would be, "Diagnose the deterioration status of the facility from the images captured by the camera and generate the necessary repair plan." The introduction of this system will improve the safety and efficiency of facility management.

[0726] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0727] Step 1:

[0728] The user uses an image acquisition device to acquire images of the target facility. The input is images of the facility's exterior and specific interior areas, generating high-resolution image data. The output is this acquired image data.

[0729] Step 2:

[0730] The user uses a communication device to send acquired image data to the server. The input is high-resolution image data, and the output is image data securely transmitted to the server. In this step, data transfer is performed using a communication protocol.

[0731] Step 3:

[0732] The server inputs the received image data into the AI ​​analysis system. The input is the transmitted image data, and the output is the analysis result of the degradation patterns contained in the image. As part of the data processing, image preprocessing is performed, and patterns such as cracks and corrosion are identified using a generated AI model.

[0733] Step 4:

[0734] Based on the analysis results, the server uses AI to identify areas requiring repair and their priority. The input is analyzed degradation data, and the output is a list of degraded areas and their priorities. A ranking is generated using a prioritization algorithm based on the severity of the degradation.

[0735] Step 5:

[0736] The server generates a reinforcement plan based on identified repair locations and their priorities. The input is a list of prioritized repair locations, and the output is a detailed reinforcement plan. The plan includes materials to be used, work processes, and schedules, and an optimization algorithm is employed.

[0737] Step 6:

[0738] The server presents the generated augmentation plan to the user. The input is the created augmentation plan, and the output is information about the augmentation plan displayed in an easy-to-understand format. Details of the plan are visualized through the user interface.

[0739] Step 7:

[0740] Users input additional information regarding reinforcement plans via a communication device and send it to the server. The input consists of supplementary information and requested modifications, while the output is updated plan information. This allows for flexible plan adjustments.

[0741] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0742] This invention combines a system that diagnoses the deterioration status of buildings and infrastructure and provides an optimal reinforcement plan with an emotion engine that recognizes the user's emotions. This system mainly consists of an image acquisition device, a server, a terminal, and an emotion engine.

[0743] The user takes a picture of the target structure using an image acquisition device such as a smartphone and sends the image data to the terminal. The terminal sends the image data to a server, which analyzes the received image using an AI model. The AI ​​identifies specific deterioration patterns from the image and quantifies the degree of deterioration.

[0744] Next, the server identifies areas requiring repair based on its degradation status and prioritizes repairs by considering multiple factors. Based on this, an optimal reinforcement plan is formulated, and the emotion engine further analyzes user reactions.

[0745] The emotion engine analyzes the user's voice and facial expressions as they review the reinforcement plan, and evaluates their emotions. This information is then used to suggest additional information and support that alleviate the user's anxiety and concerns. For example, if the emotion engine senses that the user is anxious, it can provide detailed explanations and present actual repair examples to address that anxiety.

[0746] As a concrete example, suppose an individual user uses this system to inspect the old roof of their home. The user takes photos of the roof and sends them to the server, where the AI ​​identifies deteriorated areas. The server evaluates the extent of the deterioration and creates a reinforcement plan for areas that need repair. If the user feels uneasy after reviewing the plan, the emotion engine detects this and presents successful repair case studies by roofing professionals to provide reassurance. In this way, user satisfaction can be increased by providing an integrated solution that combines technical problem diagnosis with emotional support.

[0747] The following describes the processing flow.

[0748] Step 1:

[0749] The user uses an image acquisition device, such as a smartphone, to take pictures of the building or infrastructure they want to diagnose. After taking the pictures, the image data is saved to the device.

[0750] Step 2:

[0751] The terminal compresses the stored image data, adds the necessary metadata, and sends it to the server. A secure communication protocol is used for transmission to avoid data loss.

[0752] Step 3:

[0753] The server preprocesses the received image data before inputting it into the analysis system. This preprocessing includes resizing and noise reduction, preparing the images for analysis.

[0754] Step 4:

[0755] The server inputs pre-processed image data into an AI model to analyze the degradation status. The AI ​​model identifies anomalies in the image (e.g., cracks and rust), quantifies them, and generates an evaluation result.

[0756] Step 5:

[0757] The server identifies areas requiring repair based on evaluation results obtained from the AI ​​model. Next, it considers multiple factors such as the degree of deterioration, the importance of the structure, and the user's budget to determine the priority of repairs.

[0758] Step 6:

[0759] The server creates a detailed reinforcement plan based on the prioritized results. This plan includes repair processes, materials to be used, and implementation schedules, and is compiled in an easily viewable format such as PDF or images.

[0760] Step 7:

[0761] When the terminal presents the augmentation plan received from the server to the user, the emotion engine is also activated. When the user reviews the augmentation plan, the emotion engine performs voice analysis and facial recognition to detect the user's emotions.

[0762] Step 8:

[0763] Based on the user's emotional information obtained by the emotion engine, the server provides additional information to alleviate the user's anxiety and concerns about the plan. For example, if anxiety is detected, it may present past success stories or provide more detailed explanations.

[0764] Step 9:

[0765] After receiving additional information and support, users will be able to enter questions and additional instructions regarding the reinforcement plan into the terminal. The terminal will collect this information and send it to the server in real time.

[0766] Step 10:

[0767] The server analyzes additional information received from the user and updates the reinforcement plan if necessary. The updated plan is then sent to the terminal and presented to the user, optimizing the plan.

[0768] (Example 2)

[0769] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0770] Conventional building and infrastructure deterioration assessment systems focused on technical assessments and the presentation of reinforcement plans, but lacked sufficient support that considered the user's psychological state. As a result, user anxieties and concerns about the proposed reinforcement plans were not addressed, leading to low user satisfaction with the assessment results and proposals. Furthermore, efficient information management that takes into account region-specific conditions remained a challenge.

[0771] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0772] In this invention, the server includes means for determining the deterioration status using artificial intelligence to analyze image data, means for creating and presenting reinforcement plans, emotion engine means for detecting user emotions and providing appropriate additional information, and means for analyzing user emotion data and proposing psychological support. This makes it possible to not only provide technical diagnostic results but also to provide comprehensive support that takes user emotions into consideration, thereby improving user satisfaction with deterioration diagnosis and reinforcement proposals.

[0773] An "image acquisition device" refers to equipment used to acquire image data of structures such as buildings and infrastructure. Specifically, this includes smartphones and digital cameras equipped with camera functions.

[0774] "Artificial intelligence" is a technology developed to allow computer systems to mimic human intelligence, and is primarily used for data analysis and pattern recognition.

[0775] "Deterioration status" refers to the condition or degree to which a structure has deteriorated due to aging or environmental factors, and whether repairs are necessary.

[0776] A "reinforcement plan" is a plan that outlines specific procedures and methods for repairing or improving deteriorated structures, and also includes their priorities.

[0777] An "emotion engine" refers to a technology or system that analyzes a user's voice and facial expressions to recognize their emotions and provide appropriate support or additional information.

[0778] "Prioritizing" means evaluating the importance and necessity of multiple options or tasks, and deciding the order in which to perform them.

[0779] "Psychological support" refers to activities and information provided that take emotional aspects into consideration in order to alleviate users' anxiety and concerns and provide them with a sense of security.

[0780] "Local information" refers to geographical, climatic, social, and economic data and knowledge related to a specific region, and is used for disaster response and planning.

[0781] This system consists of an image acquisition device, a server, a terminal, and an emotion engine. First, the user acquires image data of buildings and infrastructure using an image acquisition device such as a smartphone or digital camera. The image data captured by the user is sent to a terminal, such as a smartphone or personal computer. The terminal then sends this image data to the server.

[0782] The server possesses powerful computing capabilities and analyzes the received image data based on a generative AI model. This generative AI model is built using deep learning techniques to identify deterioration patterns from building images. Typical frameworks used include TensorFlow and PyTorch. The server detects signs of deterioration in the images and quantifies the degree of deterioration.

[0783] Furthermore, based on the analysis results, the server identifies where repairs are needed, prioritizes them, and creates an optimal reinforcement plan. This plan is presented to the user via their terminal. For example, it utilizes data on the extent and location of cracks in the building to specifically show effective reinforcement methods and procedures.

[0784] Subsequently, the user reviews the reinforcement plan using a device. During this process, the device captures the user's facial expressions and voice data, which are then analyzed by a server using an emotion engine. This emotion engine has the technology to identify whether the user has any anxieties or doubts about the plan. Based on the detected emotions, additional information is provided to alleviate the user's anxiety. For example, actual examples of successful reinforcements and testimonials from construction companies may be presented.

[0785] As a concrete example, consider a case where a user inspects the old roof of their home. The user takes a picture of the roof with their smartphone and sends it to a server for analysis. The server identifies the deteriorated areas, develops a reinforcement plan, and presents it. If the emotion engine detects the user's anxiety, it will show past repair examples from roofing experts to reassure the user.

[0786] An example of a prompt message input to a generated AI model is: "This program combines an AI system that diagnoses the deterioration status of a building with an emotion engine that recognizes user emotions. Based on the information obtained from image data, identify deteriorated areas, develop a reinforcement plan, and provide user support through emotion analysis."

[0787] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0788] Step 1:

[0789] The user uses an image acquisition device to capture image data of the target building or infrastructure. Specifically, they use the camera function of their smartphone to take multiple photos of the area to be diagnosed. The captured image data is saved to the smartphone's memory. The input here is optical information indicating the current state of the building, and the output is data in an image file format (such as JPEG or PNG).

[0790] Step 2:

[0791] The terminal uploads image data acquired from the user's smartphone to the server. In this process, the terminal transmits data to the server using the internet. Specifically, the user selects an image using a dedicated application and presses the send button. The input is image data stored on the terminal, and the output is image data stored on the server's storage.

[0792] Step 3:

[0793] The server analyzes the received image data using a generative AI model. The AI ​​model is trained to identify degradation patterns (cracks, distortion, discoloration, etc.). Specifically, the AI ​​analyzes the pixel information of the image, extracts features, and performs pattern recognition. The input is image data stored on the server, and the output is quantified data indicating the areas of degradation (e.g., crack width, location).

[0794] Step 4:

[0795] The server creates a prioritized reinforcement plan based on the degradation data obtained from the analysis. The server determines the priority of repairs, taking into account the severity and scope of the degradation. Specifically, it analyzes quantitative data and generates repair proposals in accordance with a reinforcement plan template. The input is quantitative data on the degraded areas, and the output is a prioritized reinforcement plan.

[0796] Step 5:

[0797] The server sends the generated augmentation plan to the user via the terminal. The user reviews and understands the augmentation plan on the terminal. Specifically, the user opens the application and views the plan file. The input is the augmentation plan (digital document), and the output is the augmentation plan provided to the user as visual information.

[0798] Step 6:

[0799] The terminal acquires user voice and facial expression data when reviewing the reinforcement plan and sends it to the server. During this process, the terminal uses its camera and microphone to record the user's reactions. The input is the user's voice and video, and the output is emotion data sent to the server.

[0800] Step 7:

[0801] The server uses an emotion engine to analyze the user's voice and facial expressions and perform an emotional assessment. The server recognizes whether the user is experiencing anxiety or concern and generates appropriate additional information. The input is emotional data, and the output is additional information designed to enhance the user's sense of security (e.g., success stories, expert comments).

[0802] Step 8:

[0803] The server provides the user with additional information based on sentiment analysis via the terminal. The user uses this information to deepen their understanding of the reinforcement plan and reduce anxiety. Specifically, the user reviews additional information such as videos and text. The input is supplementary information based on sentiment analysis, and the output is an improvement in the user's psychological sense of security.

[0804] (Application Example 2)

[0805] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0806] In diagnosing deterioration of factory equipment and infrastructure, conventional methods make it difficult to accurately determine the extent of deterioration and formulate reinforcement plans based on the results. Furthermore, there is a lack of care that addresses the anxieties and concerns felt by users. This can lead to delays in timely and appropriate maintenance, potentially reducing work efficiency.

[0807] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0808] In this invention, the server includes means for acquiring image information of an object using an image acquisition device, means for identifying the deterioration status of the object using an automatic processing device that analyzes the image information, means for identifying repair locations and setting their importance based on the identification results, means for creating an optimal reinforcement plan based on the importance, means for presenting the reinforcement plan to the user, and means for detecting the user's emotions and providing information in accordance with those emotions. This makes it possible to accurately diagnose the deterioration of factory equipment and optimize maintenance plans that take into account the user's emotions.

[0809] An "image acquisition device" is a device used to acquire image information of an object.

[0810] An "automated processing device" is a system that analyzes acquired image information to identify the degree of deterioration of an object.

[0811] "Importance" is a measure of priority assigned to the identified repair locations.

[0812] A "reinforcement plan" is a plan for optimized repair and reinforcement based on the identified deterioration status.

[0813] A "user" is a person or organization that operates the system and receives suggestions for reinforcement plans.

[0814] "Providing information tailored to emotions" refers to detecting the user's emotions and then providing additional information to alleviate their anxiety or concerns.

[0815] To implement this invention, first, a camera device capable of photographing equipment and structures within a factory is used as the image acquisition device. The user uses this camera to photograph objects that may be deteriorating. The captured image data is transmitted to the server via a terminal.

[0816] The server uses an automated processing unit, specifically an AI processing module, to analyze the transmitted image data. Through this analysis, the AI ​​system identifies the deterioration patterns of the object and quantifies the degree of deterioration. For this AI processing, for example, open-source image processing libraries such as OpenCV or deep learning frameworks such as TensorFlow can be used.

[0817] Furthermore, the server assigns a priority level to the identified repair locations and creates an optimal reinforcement plan based on this level. This reinforcement plan is created using planning software to develop a feasible maintenance plan tailored to the degree and scale of the identified deterioration.

[0818] Meanwhile, the server uses emotion recognition software to analyze the user's response to the reinforcement plan. This is done by evaluating facial expressions and voice acquired through a webcam and microphone. This emotion evaluation uses an emotion analysis engine such as EmotionEngine. Based on the results, if the user feels anxiety or concern, additional information and specific examples can be provided to support the user according to their needs.

[0819] As a concrete example, consider the case of performing regular inspections of factory equipment. When a user takes photos of the equipment and sends them to the system, the server first uses an AI model to diagnose the degree of deterioration and design a repair plan. At the same time, an emotion engine detects the user's reactions and, if necessary, presents additional information or past success stories, thereby improving the sense of security during the inspection work.

[0820] An example of a prompt to input into the generating AI model is as follows: "Please tell me how to effectively implement a degradation diagnosis support application for use in a factory. In particular, please focus on optimizing information delivery based on the worker's emotions."

[0821] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0822] Step 1:

[0823] The user uses a camera device to photograph equipment and structures within the factory and acquires the image data. This image data is transmitted to the server via a terminal. The input is the captured image data, and the output is the transmitted image data.

[0824] Step 2:

[0825] The server uses an AI model to analyze the received image data. Specifically, it uses OpenCV and TensorFlow to perform image processing, identify degradation patterns, and perform classification. The input is the image data sent in step 1, and the output is the identified degradation patterns and classification results.

[0826] Step 3:

[0827] The server sets the importance of the repair locations based on the identified degradation information. Using planning software, it lists the locations in descending order of priority, taking into account the degree of degradation. The input is the identification result from step 2, and the output is a list of repair locations with assigned priorities.

[0828] Step 4:

[0829] The server creates an optimal reinforcement plan based on a list of repair locations. This involves using a planning algorithm to develop specific maintenance procedures and schedules. The input is the priority list from step 3, and the output is a detailed reinforcement plan.

[0830] Step 5:

[0831] The server uses emotion recognition software to analyze the user's facial expressions and voice when presenting reinforcement plans. It uses emotion analysis tools such as EmotionEngine to evaluate emotions like anxiety and concern. The input is user facial expression and voice data, and the output is the result of the emotion evaluation.

[0832] Step 6:

[0833] Based on the emotional assessment results, the server provides the user with additional information and success stories to help alleviate anxiety. This helps the user accept the reinforcement plan with confidence. The input is the emotional assessment result from step 5, and the output is the additional information and success stories presented to the user.

[0834] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0835] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0836] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0837] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0838] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0839] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0840] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0841] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0842] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0843] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0844] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0845] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0846] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0847] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0848] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0849] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0850] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0851] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0852] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0853] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0854] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0855] The following is further disclosed regarding the embodiments described above.

[0856] (Claim 1)

[0857] A means for acquiring image data of a target structure using an image acquisition device,

[0858] A means for determining the deterioration status of a structure using artificial intelligence that analyzes the aforementioned image data,

[0859] A means of identifying repair locations and prioritizing them based on the assessment results,

[0860] A means for creating an optimal reinforcement plan based on the aforementioned priorities,

[0861] A system including means for presenting the aforementioned reinforcement plan to the user.

[0862] (Claim 2)

[0863] The system according to claim 1, further comprising means that enable the user to input additional information regarding the reinforcement plan using a communication device.

[0864] (Claim 3)

[0865] The system according to claim 1, further comprising means for integrating the aforementioned determination results and reinforcement plans with regional information and storing them in a database, making them available for use in disaster response planning.

[0866] "Example 1"

[0867] (Claim 1)

[0868] A means for collecting image information of the target facility using an image acquisition function,

[0869] A means for determining the extent of damage to a facility using artificial intelligence that analyzes the aforementioned image information,

[0870] A means for identifying the repair location and setting its importance based on the assessment results,

[0871] Based on the aforementioned importance level, a means for formulating an optimal repair plan,

[0872] Means for providing the aforementioned repair plan to users,

[0873] A means of improving analysis accuracy using a generative AI model,

[0874] A means of collecting additional information from users via prompt messages and incorporating it into the plan,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, which enables the user to input additional information regarding the repair plan using a communication device and to further optimize the plan using a generated AI model.

[0878] (Claim 3)

[0879] The system according to claim 1, which combines the aforementioned determination results and repair plans with geographic information and stores them in an information management device, making them available for use in emergency response planning.

[0880] "Application Example 1"

[0881] (Claim 1)

[0882] A means for acquiring image data of a target facility using an image acquisition device,

[0883] A means for determining the deterioration status of a facility using artificial intelligence that analyzes the aforementioned image data,

[0884] A means of identifying repair locations and prioritizing them based on the assessment results,

[0885] A means for creating an optimal reinforcement plan based on the aforementioned priorities,

[0886] A means of presenting the aforementioned reinforcement plan to the user,

[0887] A means of notifying of anomalies in real time using monitoring equipment,

[0888] The aforementioned monitoring equipment provides a means for diagnosing deterioration while patrolling the inside of the facility,

[0889] A system that includes this.

[0890] (Claim 2)

[0891] The system according to claim 1, further comprising means that enable the user to input additional information regarding the reinforcement plan using a communication device.

[0892] (Claim 3)

[0893] The system according to claim 1, further comprising means for integrating the aforementioned determination results and reinforcement plans with regional information and storing them in a database, making them available for use in disaster response planning.

[0894] "Example 2 of combining an emotion engine"

[0895] (Claim 1)

[0896] A means for acquiring image data of a target structure using an image acquisition device,

[0897] A means for determining the deterioration status of a structure using artificial intelligence that analyzes the aforementioned image data,

[0898] A means of identifying repair locations and prioritizing them based on the assessment results,

[0899] A means for creating an optimal reinforcement plan based on the aforementioned priorities,

[0900] A means for presenting the aforementioned reinforcement plan to the user,

[0901] A means equipped with an emotion engine that detects user emotions and provides appropriate additional information,

[0902] A system that includes means to analyze user emotional data and suggest psychological support.

[0903] (Claim 2)

[0904] The system according to claim 1, further comprising means for enabling the user to input additional information regarding the reinforcement plan using a communication device, and providing support based on the analysis results of the emotion engine.

[0905] (Claim 3)

[0906] The system according to claim 1, further comprising means for integrating the aforementioned discrimination results and reinforcement plans with regional information and storing them in a database so that they can be used for disaster response planning, and for storing them together with the sentiment analysis results.

[0907] "Application example 2 when combining with an emotional engine"

[0908] (Claim 1)

[0909] A means for acquiring image information of an object using an image acquisition device,

[0910] An automated processing device for analyzing the aforementioned image information is used to identify the deterioration status of an object,

[0911] A means for identifying repair locations and setting their importance based on the identification results,

[0912] A means for creating an optimal reinforcement plan based on the aforementioned importance,

[0913] A means of presenting the aforementioned reinforcement plan to the user,

[0914] A system that includes means for detecting a user's emotions and providing information in accordance with those emotions.

[0915] (Claim 2)

[0916] The system according to claim 1, further comprising means that enable the user to input additional information regarding the reinforcement plan using a communication device.

[0917] (Claim 3)

[0918] The system according to claim 1, further comprising means for integrating the identification results and reinforcement plans with regional information and storing them in an information storage device, making them available for use in emergency response planning. [Explanation of symbols]

[0919] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring image data of a target structure using an image acquisition device, A means for determining the deterioration status of a structure using artificial intelligence that analyzes the aforementioned image data, A means of identifying repair locations and prioritizing them based on the assessment results, A means for creating an optimal reinforcement plan based on the aforementioned priorities, A system including means for presenting the aforementioned reinforcement plan to the user.

2. The system according to claim 1, further comprising means that enable the user to input additional information regarding the reinforcement plan using a communication device.

3. The system according to claim 1, further comprising means for integrating the aforementioned determination results and reinforcement plans with regional information and storing them in a database, making them available for use in disaster response planning.

Images