system

Abstract

Provide a system. [Solution] means for receiving an image of the product before repair taken by a user and storing the image in a database; a means for receiving a post-repair image taken by a user after the repair is completed and storing the image in a database; A means of using AI to compare before and after repair images and determine the legitimacy of the repair; The system includes a means for notifying the user of the determination result.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When requesting car repairs, many customers have no choice but to trust the repair company. Because there is no easy way for customers to verify the quality and legitimacy of the repairs, there is a risk of inaccurate repairs or overcharging. Therefore, a system is needed that allows everyone to request repairs with peace of mind. [Means for solving the problem]

[0005] The present invention is a system that stores images of a vehicle taken by a user before requesting repairs in a database and compares them with images taken again after the repair is completed. Specifically, it provides a means for receiving and saving images taken by the user before repair and a means for receiving and saving images after repair is completed. The system also includes a means for comparing the images before and after repair using AI, determining the legitimacy of the repair, and notifying the user of the determination result. This system deters fraudulent behavior by repairers and allows the client to objectively confirm the legitimacy of the repair.

[0006] "User" refers to an individual or corporation requesting vehicle repairs.

[0007] A "pre-repair image" refers to an exterior image of a vehicle taken by a user before the vehicle is repaired.

[0008] "Post-repair image" refers to an exterior image of the vehicle taken by the user after the vehicle has been repaired.

[0009] "Database" refers to a system for storing and managing various data, including images before and after repair.

[0010] "AI" refers to an artificial intelligence algorithm or program that automatically analyzes images and determines whether a repair is warranted.

[0011] "Decision result" refers to information indicating whether the repair is legitimate based on the results of image comparison by AI.

[0012] A "cloud server" refers to a server that receives, stores, and processes image data sent by users.

[0013] "Notification" refers to communication methods such as email or push notification to inform the user of the judgment results.

[0014] "Repairer" means any person or entity that repairs vehicles. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] In this invention, a user first takes a photo of the exterior of their vehicle before requesting repairs and uploads the image data to a cloud server. The cloud server then stores the received image data in a database, generates a unique "request ID," and returns it to the user. This request ID is then associated with the post-repair image.

[0037] After the repair is complete, the user takes another photo of the vehicle's exterior and uploads the image data to the cloud server. The cloud server receives the post-repair image data, associates it with the request ID generated earlier, and stores it in a database.

[0038] Next, the cloud server uses AI to compare the before and after images. The AI ​​analyzes the images and evaluates the changes in the repaired area and the quality of the repair. For example, it checks whether a dent in the door before the repair is gone after the repair, and whether the paint is applied properly.

[0039] After the pass / fail result is obtained, the cloud server records the result in a database and notifies the user via email or push notification on a dedicated app.

[0040] Program processing description

[0041] 1. The user takes a photo of the car's exterior using a smartphone.

[0042] 2. The device (user's smartphone) sends an image upload request to the cloud server from a dedicated app.

[0043] 3. The server saves the received image data, generates a unique request ID, and returns it to the user.

[0044] Next, after the repair is completed, the procedure will proceed as follows:

[0045] 1. The user takes another photo of the exterior of the vehicle after repairs.

[0046] 2. The device sends a request to upload images of the repaired item to the cloud server.

[0047] 3. The server stores the received repaired images in a database and associates them with the request ID.

[0048] Next, a comparison of the before and after images begins.

[0049] 1. The server sends image data before and after repair to the AI ​​engine.

[0050] 2. AI performs image analysis to evaluate changes in the repair area and the quality of the repair.

[0051] Finally, a judgment result is generated and notified to the user.

[0052] 1. The server receives the AI ​​analysis results and stores them in a database.

[0053] 2. The server notifies the user of the result of the judgment, for example, "the repair is valid" or "the repair is incorrect."

[0054] 3. The user receives a notification via a dedicated app or email and confirms that the repair is legitimate.

[0055] Specific examples

[0056] Uploading images before requesting repair

[0057] The user takes a photo of the left side of the car using a smartphone.

[0058] The device uploads images to the cloud using a dedicated app.

[0059] The server receives the image and stores it in the database with "Request ID 12345." The Request ID is returned to the user.

[0060] Uploading images after repair is complete

[0061] The user re-photographs the left side of the repaired car.

[0062] The device uploads images of the repaired item to the cloud.

[0063] The server receives the image and stores it, linking it to request ID 12345.

[0064] Image judgment by AI

[0065] The server sends before and after images of the repair for request ID 12345 to the AI ​​engine.

[0066] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[0067] Notification of the results

[0068] The server stores the determination result that "the repair is legitimate" in a database.

[0069] The server will notify the user of the results via email or in-app push notification.

[0070] The user receives a notification and checks the details in the app.

[0071] The above is a specific embodiment of this system.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] Before requesting repairs, the user takes a photo of the vehicle's exterior using the camera on their smartphone.

[0075] Step 2:

[0076] The device (user's smartphone) launches a dedicated app and sends a request to upload the captured images to a cloud server.

[0077] Step 3:

[0078] The server receives the uploaded image data and stores it in a database. At this time, it generates a new unique "request ID" and returns it to the user. The request ID is used to associate the before and after images of the repair.

[0079] Step 4:

[0080] After the repairs are completed, the user again uses the smartphone to take a photo of the exterior of the vehicle after repairs.

[0081] Step 5:

[0082] The device sends a request to upload the repaired image to a cloud server via a dedicated app.

[0083] Step 6:

[0084] The server receives the repaired image data and stores it in the database, linking it to the request ID generated earlier.

[0085] Step 7:

[0086] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine uses image analysis technology to evaluate the changes in the repaired area and the quality of the repair. For example, it checks whether dents have been repaired and whether the paint has been applied properly.

[0087] Step 8:

[0088] The AI ​​returns the analysis results and determines whether the repair is legitimate or illegitimate, outputting the results as a clear status such as "legitimate" or "incorrect."

[0089] Step 9:

[0090] The server stores the judgment results received from the AI ​​in a database. These judgment results are assigned a request ID.

[0091] Step 10:

[0092] The server then begins the process of notifying the user of the result of the judgment, specifically by sending an email or a push notification to a dedicated app.

[0093] Step 11:

[0094] Users receive notifications and can view detailed results within the app, allowing them to objectively determine whether the repair was legitimate.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] There is a need for a system that allows users to easily check whether their car repairs have been performed properly. Conventional methods require direct visual inspection of the condition before and after repairs, which takes time and effort. In addition, users have no way to objectively evaluate the quality of the repairs, so they may not notice if the repairs have been performed improperly. A system that solves these problems is needed.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in storage, means for receiving post-repair images taken by the user after the repair is completed and saving the images in storage, means for analyzing the pre-repair and post-repair images using a generative AI model to determine the legitimacy of the repair, and means for notifying the user of the determination result, thereby enabling the user to quickly and accurately confirm whether the car repair was performed appropriately.

[0100] A "pre-repair image" is digital image data of the exterior of a vehicle taken by a user before repairs are made.

[0101] "Post-repair image" is digital image data of the exterior of the vehicle taken by the user after repairs have been completed.

[0102] "Storage" refers to storage devices and cloud services for storing digital data.

[0103] A "generative AI model" is an artificial intelligence model trained using machine learning techniques and used to perform a specific task, such as image analysis or judgment.

[0104] A "server" is a computer system that receives requests over a network, stores and processes data, and communicates with other computers.

[0105] "User" refers to an individual or company that uses this system to request automobile repairs.

[0106] "Analysis" is the process of using AI models to detect, compare, and evaluate the content of digital images.

[0107] The "judgment result" is information regarding the legitimacy of the repair obtained as a result of analysis by the generative AI model.

[0108] "Notification" refers to information sent from the system to the user, including emails and app push notifications.

[0109] The system of the present invention allows users to upload images taken before and after requesting a car repair to a cloud server, and then uses a generative AI model to determine the legitimacy of the repair. A specific embodiment of this system is described below.

[0110] First, the user uses a smartphone (e.g., iPhone (registered trademark) 13 Pro) to take a photo of the vehicle's exterior before repair. The user then selects the captured image using a dedicated app (e.g., CarRepairUploader) and presses the upload button. The device then sends this image data to a cloud server via the Internet. The cloud server receives the image data and stores it in storage (e.g., an Amazon S3 bucket). It then generates a unique "request ID" (e.g., E12345) and returns it to the user.

[0111] Next, after the repair is complete, the user takes a photo of the vehicle's exterior after repairs using the same procedure. The user then uses the dedicated app again to select the post-repair image and press the upload button. The device then sends the post-repair image data to the cloud server. The cloud server again receives the image data, associates it with the request ID (e.g., E12345) generated earlier, and stores it in storage.

[0112] The cloud server sends the image data before and after repair to a generative AI model (e.g., Amazon Rekognition). The generative AI model analyzes the images and evaluates changes in the repair area and the quality of the repair. After obtaining a judgment result, the cloud server stores the result in a database (e.g., Amazon RDS) and notifies the user. Notifications are sent via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging). Users receive a notification and can view detailed information in a dedicated app.

[0113] Specific examples

[0114] As a concrete example, the image upload before requesting repair is shown below.

[0115] Uploading images before requesting repair

[0116] The user takes a photo of the left side of the car using a smartphone (iPhone 13 Pro).

[0117] The device (iPhone 13 Pro) uploads images to the cloud server using a dedicated app (CarRepairUploader).

[0118] The server receives the image and stores it in storage (Amazon S3 bucket) with "Request ID E12345." The Request ID is returned to the user.

[0119] Uploading images after repair is complete

[0120] The user takes another photograph of the left side of the car after repairs.

[0121] The device (iPhone 13 Pro) also uploads images of the repaired item to a cloud server using a dedicated app (CarRepairUploader).

[0122] The server receives the image, associates it with the request ID E12345, and stores it in storage.

[0123] Image judgment by AI

[0124] The server sends the before and after images of the repair for request ID E12345 to the generated AI model (Amazon Rekognition).

[0125] The AI ​​engine analyzes the images and checks for changes in the repaired area, for example, to see if a dent in the door before the repair has disappeared after the repair.

[0126] Notification of the results

[0127] The server stores the determination that the repair is valid in a database (Amazon RDS).

[0128] The server notifies the user of the result, for example via email (SendGrid) or in-app push notification (Firebase Cloud Messaging).

[0129] The user receives a notification and checks the details using a dedicated app (CarRepairUploader).

[0130] Using generative AI models, the validity of vehicle repairs can be efficiently and accurately assessed. This system provides users with a means to objectively verify the quality of repairs and can prevent inappropriate repairs.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] The user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then opens a dedicated app (e.g., CarRepairUploader), selects the image, and presses the upload button.

[0134] Input: High-resolution image data before repair taken with a smartphone

[0135] How it works: The user selects an image in the app and taps the upload button.

[0136] Output: The captured image data is saved on the device and an upload request is sent to the server.

[0137] Step 2:

[0138] The device sends the captured image data to a cloud server via the Internet.

[0139] Input: Image data before repair and request to upload to server

[0140] How it works: Your device uses its internet connection to send captured images to a cloud server.

[0141] Output: The image data arrives at the cloud server.

[0142] Step 3:

[0143] The server stores the received image data in storage (e.g., Amazon S3 bucket) and generates a unique "request ID" (e.g., E12345). The server returns the generated request ID to the user.

[0144] Input: Image data before repair submitted

[0145] Operation: Image data is stored in Amazon S3, a unique ID generation algorithm is used to generate a request ID, and the generated request ID is returned to the device.

[0146] Output: Returns the image data stored in the storage and the request ID.

[0147] Step 4:

[0148] After the repair is complete, the user takes another photo of the vehicle's exterior with their smartphone, selects the image after repair using the dedicated app, and presses the upload button.

[0149] Input: High-resolution image data of the repaired item taken with a smartphone

[0150] How it works: The user selects an image in the app and taps the upload button.

[0151] Output: The captured image data is saved on the device and an upload request is sent to the server.

[0152] Step 5:

[0153] The device then sends the repaired image data back to the cloud server via the Internet.

[0154] Input: Repaired image data and upload request to the server

[0155] How it works: Your device uses its internet connection to send captured images to a cloud server.

[0156] Output: The image data arrives at the cloud server.

[0157] Step 6:

[0158] The server stores the received repaired image data in storage (e.g., Amazon S3 bucket) and associates it with the request ID (e.g., E12345) generated earlier.

[0159] Input: The image data after repair and the existing request ID

[0160] Operation: The repaired image data is saved in Amazon S3 and linked to the existing request ID.

[0161] Output: Image data stored in storage and ID link information

[0162] Step 7:

[0163] The server sends the image data before and after the repair to a generative AI model (e.g., Amazon Rekognition), which analyzes the images and evaluates the changes in the repaired area and the quality of the repair.

[0164] Input: Before and after image data

[0165] How it works: The server sends image data to the generative AI model, which then analyzes the image.

[0166] Output: Evaluation results regarding changes in repaired areas and repair quality

[0167] Step 8:

[0168] The server receives the analysis results from the AI ​​engine, stores them in a database (e.g., Amazon RDS), and notifies the user of the results via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging).

[0169] Input: Analysis results from a generative AI model

[0170] Behavior: The analysis results are inserted into a database and the results are notified to the user using an email sending API or push notification API.

[0171] Output: Analysis results stored in a database and notifications to the user

[0172] Step 9:

[0173] Users will receive a notification and can check detailed information in a dedicated app.

[0174] Input: Notification of judgment results by email or push notification

[0175] How it works: The user opens the app and taps the "View Repair Results" button to view the results.

[0176] Output: Repair assessment results displayed in the user's app

[0177] The above is the flow of the program processing of the present invention.

[0178] (Application example 1)

[0179] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0180] Conventional technology requires users to manually take photos of the vehicle's exterior and upload the images before and after repair to a cloud server, which can lead to manual labor and errors. Furthermore, there is a lack of a means to objectively evaluate the legitimacy of repairs, making reliable damage diagnosis difficult. This makes it difficult to control the quality of repairs and achieve efficient maintenance.

[0181] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0182] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, means for notifying the user of the determination result, means for automatically photographing the vehicle exterior using an on-board camera, means for uploading the image data to a cloud server, means for the cloud server to generate a unique request ID, and means for saving the photographed images in a database based on the generated request ID. This enables automatic photographing of the vehicle exterior and uploading the images to the cloud server, allowing AI to determine whether there is damage, enabling efficient and reliable damage diagnosis.

[0183] "User" refers to a person who uses the system to request repairs or inspections of a vehicle, or the owner of such a vehicle.

[0184] The "image before repair" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device before the vehicle is repaired.

[0185] A "database" is a system for accumulating, storing, and managing received image data and related information.

[0186] "Post-repair image" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device after the vehicle has been repaired.

[0187] "AI" refers to artificial intelligence technology, a general term for programs and algorithms that have the ability to analyze image data and compare before and after repairs.

[0188] A "cloud server" is a physical or virtual server used over a network that provides resources for storing and processing image data and related information.

[0189] A "request ID" is a unique identification code generated by the cloud server that is linked to a specific repair request.

[0190] "Photography device" refers to a camera or other photographic equipment that is mounted on a vehicle and automatically photographs the exterior of the vehicle.

[0191] The "judgment result" indicates the result of image analysis by AI, an evaluation of the validity of the repair and the presence or absence of damage, and its conclusion.

[0192] The "notification means" is a function for notifying the user of the judgment result, and includes email, push notification, etc.

[0193] This invention relates to a system that automatically diagnoses damage to autonomous vehicles. It eliminates the need for users to manually upload images of the vehicle before and after repairs, and also enables automatic evaluation of the quality of repairs by performing image analysis.

[0194] The system consists of the following main components:

[0195] 1. Vehicle-mounted imaging equipment:

[0196] The camera system installed in the vehicle automatically captures the exterior of the vehicle, mainly before the vehicle departs and after it returns, eliminating the need for the user to manually take photos.

[0197] 2. Cloud Server:

[0198] The captured image data is uploaded to a cloud server, which stores the received image data in a database and generates a unique request ID. The generated request ID is linked to the images before and after repair.

[0199] 3. AI image analysis engine:

[0200] AI analyzes the image data before and after repair stored on a cloud server. This AI uses image analysis technology to evaluate changes in the repaired area and the quality of the repair. Specifically, it checks whether dents and scratches that existed before the repair have been properly repaired and whether the paint has been applied accurately.

[0201] 4. Notification system:

[0202] The AI's assessment results are stored on a cloud server and notified to users and fleet managers. This notification is mainly via email or push notification, allowing users to immediately confirm the legitimacy of repairs.

[0203] Hardware and software used

[0204] Camera system: Used to capture the exterior of the vehicle. High-resolution cameras are recommended.

[0205] Cloud server: A general cloud server such as AWS (registered trademark) or Google (registered trademark) Cloud is used.

[0206] Database system: A relational database such as MySQL (registered trademark) or PostgreSQL is used.

[0207] AI image analysis software: Deep learning frameworks such as TENSORFLOW (registered trademark) and PyTorch are used.

[0208] Specific examples

[0209] A specific example of how this system can be used is shown below.

[0210] Photograph and upload before requesting repair:

[0211] The vehicle automatically takes a photo of the exterior before departing.

[0212] The vehicle system uploads the image data to a cloud server.

[0213] The cloud server receives the image data and stores it in the database as request ID 12345.

[0214] Photograph and upload after repair is complete:

[0215] After the vehicle returns from repairs, the exterior is automatically photographed again.

[0216] The vehicle system uploads post-repair image data to a cloud server.

[0217] The cloud server receives the image data and stores it, linking it to request ID 12345.

[0218] AI image assessment:

[0219] The cloud server sends image data before and after repair for request ID 12345 to the AI ​​engine.

[0220] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[0221] Notification of decision:

[0222] The cloud server stores the determination result that "the repair is legitimate" in a database.

[0223] The cloud server notifies the user of the results via email or in-app push notification.

[0224] The user receives a notification and checks the details in the app.

[0225] Prompt Sentence Examples

[0226] Develop an application that allows users to take photos of the exterior of autonomous vehicles using a smartphone and upload the images to the cloud. The application should have the ability to use AI to compare before and after images to determine the quality of the repair.

[0227] The above is a specific embodiment for carrying out the present invention.

[0228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0229] Step 1:

[0230] A camera mounted on the vehicle automatically photographs the vehicle's exterior. The images are captured by a high-resolution camera, and metadata such as location and timestamp are also saved. (Input): Vehicle exterior. (Output): High-resolution image.

[0231] Step 2:

[0232] The device uploads the captured image data to the cloud server. At this time, the vehicle ID and other identification information are also sent. (Input): Captured image data and related identification information, (Output): Image data uploaded to the cloud server.

[0233] Step 3:

[0234] The cloud server stores the received image data in a database and generates a unique request ID, which makes it easier to identify images before and after repair. (Input): Received image data and identification information, (Output): Image data stored in the database and generated request ID.

[0235] Step 4:

[0236] After the user requests repairs, the vehicle automatically photographs its exterior once the repairs are complete. The photographic device captures the exterior of the vehicle after repairs and generates image data again. (Input): Appearance of the repaired vehicle, (Output): High-resolution image after repairs.

[0237] Step 5:

[0238] The terminal re-uploads the repaired image data to the cloud server. At this time, identification information including the vehicle ID and request ID is also sent. (Input): Repaired image data and related identification information, (Output): Upload of repaired image data to the cloud server.

[0239] Step 6:

[0240] The cloud server receives the image data after repair and stores it in a database, linking it to the request ID already saved. (Input): Received image data after repair and request ID, (Output): Image data after repair saved in the database.

[0241] Step 7:

[0242] The cloud server sends the image data before and after repair to an AI image analysis engine. This AI engine performs image analysis using generative AI models such as TensorFlow and PyTorch. (Input): Image data before and after repair. (Output): Analysis results regarding the quality of the repair.

[0243] Step 8:

[0244] The AI ​​image analysis engine compares image data before and after repair, assessing changes in each part and the quality of the repair. It determines whether the changes are appropriate and concludes that the repair is appropriate or inaccurate. (Input): Image data before and after repair, (Output): Evaluation results regarding the quality of the repair.

[0245] Step 9:

[0246] The server stores the AI ​​analysis results in a database and notifies the user. Notifications are mainly sent via email or push notifications. (Input): AI analysis results, (Output): Notification to the user.

[0247] Step 10:

[0248] The user receives a notification and can check the detailed repair quality results in the app, which allows them to know whether the repair was justified or inappropriate. (Input): Notification from the server, (Output): Display of analysis results in the user's app.

[0249] This series of processes not only allows users to easily and accurately grasp the condition of their vehicle before and after repairs, but also allows them to use the data to evaluate the quality of repairers.

[0250] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0251] The present invention provides a system that combines a process in which a user takes exterior images of a vehicle before and after requesting repairs and uploads them to a cloud server with an emotion engine that recognizes the user's emotions.

[0252] System Configuration

[0253] 1. The user uses their smartphone to take a photo of the car's exterior before repairs and uploads the image to a cloud server via a dedicated app. At this point, the emotion engine recognizes the user's emotional state and sends that data along with the image.

[0254] 2. The device (user's smartphone) generates a request to send the captured image and the user's emotional data to the cloud server and sends it.

[0255] 3. The server stores the received image data and emotion data in a database, generates a unique "request ID," and returns it to the user. The request ID is used to associate the images and emotion data before and after repair.

[0256] The procedures after the repair is completed are almost the same as those described above.

[0257] 4. After the repair is complete, the user again uses their smartphone to take a photo of the vehicle's exterior and uploads the image to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state and sends that data along with the image.

[0258] 5. The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[0259] 6. The server receives the repaired image data and emotion data, associates them with the request ID, and stores them in a database.

[0260] 7. The server uses AI to compare before and after images and evaluate the changes and quality of the repair, such as whether the dent in the door was fixed or whether the paint was applied properly.

[0261] 8. The AI ​​generates the results of the image analysis and returns them to the server, which stores them in a database.

[0262] 9. The server then initiates the procedure to notify the user of the result of the judgment. At this time, the emotion engine checks the user's emotional state before the notification and selects an appropriate notification method based on the results. For example, if the user is in an unstable emotional state, the content of the notification can be flexibly adjusted.

[0263] Specific examples

[0264] Uploading images before requesting repair

[0265] The user takes a photo of the left side of the car using their smartphone and uploads the image to a cloud server via a dedicated app. The emotion engine recognizes the user's emotions and sends emotional data such as "relief" along with the image.

[0266] The device sends a request to the cloud server. The server receives the image and emotion data and stores it in the database under "Request ID 12345."

[0267] Uploading images after repair is complete

[0268] The user then takes another photo of the left side of the repaired car and uploads it to the cloud server via a dedicated app. The emotion engine then recognizes the user's emotions again, and includes emotional data such as "expectation."

[0269] The device sends a request to the cloud server. The server receives the image and emotion data, associates it with request ID 12345, and stores it.

[0270] AI-based image judgment and emotional response

[0271] The server sends the image data before and after the repair to the AI ​​engine for analysis. The AI ​​evaluates the quality of the repair and generates a judgment that the repair is legitimate.

[0272] The server receives the judgment result and stores it in a database. The emotion engine checks the user's latest emotional state and selects an appropriate notification method (e.g., a soft-spoken notification message).

[0273] The server sends the result of the assessment to the user via email or push notification, which includes the assessment result and additional reassurance information according to the user's emotional state.

[0274] The system allows users to ensure the validity of repairs and also provides emotional consideration during the process.

[0275] The processing flow will be explained below.

[0276] Step 1:

[0277] The user uses a smartphone to take a photo of the car's exterior before repairs are made, and the image is then uploaded to a cloud server using a dedicated app.

[0278] Step 2:

[0279] The device generates and sends a request to upload the pre-repair image to a cloud server via a dedicated app. It also analyzes the user's emotional state (e.g., "relief" or "anxiety") through an emotion engine and sends that data together.

[0280] Step 3:

[0281] The server receives the uploaded image of the item before repair and the user's emotion data, stores them in a database, and generates a new "request ID" and returns it to the user.

[0282] Step 4:

[0283] After the repair is complete, the user takes another photo of the exterior of the car using their smartphone.

[0284] Step 5:

[0285] The device generates and sends a request to upload the repaired image and the user's emotion data (analyzed by the emotion engine) to the cloud server.

[0286] Step 6:

[0287] The server receives the post-repair image and emotion data and stores them in the database with the same request ID as the pre-repair image.

[0288] Step 7:

[0289] The server sends the image data before and after the repair to the AI ​​engine and requests image analysis, which then begins the process of evaluating changes in the repaired area and the quality of the repair.

[0290] Step 8:

[0291] The AI ​​engine analyzes images to determine whether repairs are warranted, for example, checking that a dent in a door has been repaired or that the paint is properly applied.

[0292] Step 9:

[0293] The AI ​​engine generates a verdict and returns the result to the server, for example, "The repair is valid" or "The repair is incorrect."

[0294] Step 10:

[0295] The server stores the received judgment results in a database. At this time, the emotion engine checks the user's latest emotional state and determines how to notify the judgment results based on that result.

[0296] Step 11:

[0297] The server then initiates a procedure to notify the user of the result of the assessment. For example, if the user is in an anxious emotional state, the server may soften the content of the notification.

[0298] Step 12:

[0299] The user receives the results via a dedicated app or email, which includes additional reassurance information tailored to the user's emotional state.

[0300] Specific examples

[0301] Uploading images before requesting repair

[0302] The user takes a photo of the left side of the car with their smartphone. The app prepares the image for upload to a cloud server. The emotion engine analyzes the user's emotion as "relief" and sends the image to the server.

[0303] The device (smartphone) sends images and emotional data to a cloud server.

[0304] Uploading repaired images

[0305] The user takes another photo of the left side of the car after the repair is complete. The app prepares to upload the repaired image to the cloud server. The emotion engine analyzes the user's emotion as "expectation" and sends it to the server.

[0306] The device (smartphone) sends images of the repaired item and emotional data to a cloud server.

[0307] AI-based image judgment and emotional response

[0308] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine performs the image analysis and generates a result that the repair is legitimate.

[0309] The server stores the result in a database and uses the emotion engine to check the user's current emotional state, which determines that the user needs to be notified in a softer tone.

[0310] The server sends the result of the assessment via email or push notification to a dedicated app, which includes additional information to reassure the user along with the result.

[0311] Users receive a notification and can check the detailed results in a dedicated app. The notification is designed with emotional consideration, giving them peace of mind regarding the repair details.

[0312] The above is a specific embodiment of the invention in which an emotion engine is combined.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] While conventional car repair evaluation systems can judge the legitimacy of repairs by comparing before and after images, they lack a notification method that takes into account the emotional state of the repair requester, which has led to issues with not being able to fully achieve the requester's satisfaction or sense of security. Furthermore, conventional systems do not use emotional data for evaluation, and so the evaluation of repair companies remains one-sided.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0317] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for recognizing the user's emotional state and transmitting emotional data along with the images, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, and means for selecting an appropriate notification method and notifying the user based on the determination result and the user's latest emotional state. This not only makes it possible to objectively evaluate the legitimacy of the repair, but also increases user satisfaction and peace of mind by adopting a notification method that takes the user's emotional state into consideration.

[0318] "User" refers to any individual or corporation that uses this system.

[0319] A "terminal" is a device used by a user, such as a smartphone or tablet, that is primarily used for taking photos and transmitting data.

[0320] "Server" refers to a central processing unit that manages and stores received data and performs AI analysis and notification processing.

[0321] A "cloud server" is a server used over the Internet that includes resources for storing, processing, and analyzing data.

[0322] A "database" refers to a system that centrally manages information and allows it to be stored, searched, and updated.

[0323] An "emotion engine" refers to software or algorithms that recognize a user's emotional state and process it as data.

[0324] "Image data" refers to images of the vehicle before and after repair that are taken by the user and uploaded to the cloud server.

[0325] "Emotion data" refers to data that indicates the user's emotional state as recognized by the emotion engine.

[0326] "Request ID" refers to a unique identifier generated to identify and manage a series of data related to a repair request.

[0327] "AI engine" refers to the artificial intelligence algorithm used to analyze before and after repair images and evaluate the quality of the repair.

[0328] "Judgment result" refers to the evaluation result obtained by the AI ​​engine by comparing and analyzing images before and after repair.

[0329] "Notification means" refers to a function that sends information to the user in an appropriate manner based on the judgment result and emotion data.

[0330] A "transaction" refers to a mechanism for processing a series of operations performed within a database as a single unit and maintaining its consistency.

[0331] "HTTPS protocol" refers to an internet protocol for secure communications.

[0332] MODE FOR CARRYING OUT THE INVENTION

[0333] This invention is a system that records the process in detail when a user requests car repairs, and evaluates the quality of the repair based on images taken before and after the repair and the user's emotional state. This system is composed of the user's device, a cloud server, a database, an AI engine, and an emotion engine.

[0334] Hardware and software used

[0335] Device: A smartphone or tablet operated by the user

[0336] Cloud server: A server that stores, processes, and analyzes data (e.g., AWS, Azure (registered trademark))

[0337] Database: A system for storing image data and emotion data (e.g., MySQL, PostgreSQL)

[0338] Emotion engine: Software for recognizing user emotions (e.g., Emotion API, IBM Watson® Tone Analyzer)

[0339] AI engine: Algorithm that analyzes before and after repair images and evaluates the quality of the repair (e.g., TensorFlow, PyTorch)

[0340] System details

[0341] Before requesting car repairs, a user takes a photo of the vehicle's condition using their smartphone and uploads the image to a cloud server using a dedicated app. At this time, the emotion engine recognizes the user's emotional state (e.g., "relief") and sends that data along with the image.

[0342] The device generates a request to send the captured image data and the user's emotion data to the cloud server, and sends it to the cloud server using the HTTPS protocol.

[0343] The server stores the received images and emotion data in a database and generates a unique request ID, which is used to associate the images and emotion data before and after repair.

[0344] After the repair is complete, the user takes another photo of the vehicle's condition with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state (e.g., "expectation") and sends that data along with the image.

[0345] The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[0346] The server receives the post-repair images and emotion data, associates them with the request ID, and stores them in a database. It then sends the pre- and post-repair image data to the AI ​​engine, requesting it to evaluate the quality of the repair.

[0347] The AI ​​engine compares image data before and after the repair, assesses changes in the repaired area, and evaluates the quality of the repair. The results are sent back to the server and stored in a database.

[0348] The emotion engine checks the user's latest emotional state before notifying them of the evaluation results and selects an appropriate notification method (e.g., soft-spoken text).

[0349] The server generates a message to notify the user based on the judgment result and emotion data, and sends it via email or push notification.

[0350] Specific examples

[0351] For example, if a user takes a photo of the left side of a car using a smartphone and uploads the image to a cloud server using a dedicated app, the emotion engine will recognize the user's emotion as "relief" and send that data along with the image.

[0352] The device sends a request to the cloud server, and the server receives the image and emotion data and stores it in the database with request ID "12345."

[0353] After the repair is complete, the user takes another photo of the left side of the vehicle with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotion as "expectation" and sends the data along with it.

[0354] The device sends a request to the cloud server, and the server receives the image and emotion data, associates it with the request ID "12345," and stores it.

[0355] The server sends the image data before and after the repair to the AI ​​engine, which evaluates the quality of the repair and generates a result that the repair is valid. The server receives the result, and the emotion engine checks the user's latest emotional state and creates a soft-spoken notification message.

[0356] The server will send an email or push notification stating that the repair is legitimate, along with information that will reassure the user.

[0357] Prompt Sentence Examples

[0358] "The user takes a photo of the vehicle before repair and uploads it to the cloud server via a dedicated app. At this time, the emotion engine recognizes the user's emotional state and sends data indicating a sense of security."

[0359] The cloud server receives the image and emotion data, generates a request ID, and returns it to the user.

[0360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0361] Step 1:

[0362] A user uses a smartphone to take a picture of the car before repair.

[0363] Input: Use the camera function of your smartphone to take a photo of the current state of the car.

[0364] Output: Vehicle exterior image data (e.g. JPEG file).

[0365] Specific operation: The user launches the app and uses the camera function to take a photo of the area to be repaired, such as the left side of the vehicle.

[0366] Step 2:

[0367] The terminal acquires image data and the user's emotional state.

[0368] Input: Image data of the vehicle and the user's facial expression captured in real time by the camera.

[0369] Output: Vehicle image data, emotion data (e.g., relief, expectation).

[0370] Specific operation: The device's emotion engine recognizes the user's face and analyzes emotions from facial expression data. For example, it uses the Emotion API to measure feelings of relief.

[0371] Step 3:

[0372] The terminal generates and transmits a request for transmitting image data and emotion data to a cloud server.

[0373] Input: Vehicle image data, emotion data, user ID, timestamp.

[0374] Output: HTTPS outgoing request to cloud server.

[0375] Specific operation: The device generates an HTTPS request and sends it to the API endpoint of the cloud server, including image data and emotion data.

[0376] Step 4:

[0377] The server stores the received image data and emotion data in a database and generates a unique request ID.

[0378] Input: Image data, emotion data, user ID, and timestamp sent to the cloud server.

[0379] Output: The database entry and the generated request ID.

[0380] Specific operation: The server starts a transaction, records the received data in the database, and simultaneously generates a request ID and returns it to the user.

[0381] Step 5:

[0382] The server returns the request ID to the user.

[0383] Input: Generated request ID.

[0384] Output: Request ID notification to the user's device.

[0385] Specific behavior: The server generates a response containing the request ID and sends it back to the user's device via HTTPS. A notification is displayed in the user's app.

[0386] Step 6:

[0387] After the repair is completed, the user takes another picture of the repaired vehicle using their smartphone.

[0388] Input: What the vehicle will look like after repairs are complete.

[0389] Output: Image data of the vehicle's exterior after repair (e.g., JPEG file).

[0390] Specific operation: The user again uses the camera function to take a picture of the condition of the vehicle after repairs.

[0391] Step 7:

[0392] The terminal acquires the image data after repair and the user's emotional state again.

[0393] Input: A captured image of the vehicle after repair, and a user's facial expression capture.

[0394] Output: Repaired image data, emotion data (e.g., expectation).

[0395] Specific operation: The device's emotion engine performs facial recognition and analyzes and generates emotion data again.

[0396] Step 8:

[0397] The terminal generates and transmits a request for transmitting the post-repair image and emotion data to a cloud server.

[0398] Input: Repaired image data, emotion data, user ID, request ID, timestamp.

[0399] Output: HTTPS outgoing request to cloud server.

[0400] Specific operation: The device generates another HTTPS request and sends the image and emotion data to the cloud server.

[0401] Step 9:

[0402] The server receives the repaired image data and emotion data, links them to the request ID, and stores them in a database.

[0403] Input: Repaired image data, emotion data, and request ID sent to the cloud server.

[0404] Output: Updated database entry.

[0405] Specific operation: The server associates the received data based on the request ID and stores it in the database. Data integrity is ensured using transactions.

[0406] Step 10:

[0407] The server sends the image data before and after the repair to the AI ​​engine and asks it to evaluate the quality of the repair.

[0408] Input: Image data before and after repair.

[0409] Output: Data analysis request to the AI ​​engine.

[0410] Specific operation: The server accesses the AI ​​engine's API, sends image data before and after repair, and requests an evaluation.

[0411] Step 11:

[0412] The AI ​​engine analyzes the image data and evaluates changes in the repair area and the quality of the repair.

[0413] Input: Image data before and after repair.

[0414] Output: Evaluation result data (e.g., "Repair is justified").

[0415] How it works: The AI ​​engine applies image analysis algorithms to evaluate changes between images to determine the quality of the repair, and generates an assessment result that is sent back to the server.

[0416] Step 12:

[0417] The server receives the evaluation results from the AI ​​engine and stores them in a database.

[0418] Input: Evaluation result data sent from the AI ​​engine.

[0419] Output: Evaluation results stored in a database.

[0420] Specific operation: The received evaluation results are associated with the request ID and stored in the database.

[0421] Step 13:

[0422] The emotion engine checks the user's latest emotional state and selects the appropriate notification method.

[0423] Input: The user's latest emotion data.

[0424] Output: Select an appropriate notification method (e.g., a soft-spoken notification).

[0425] Specific operation: The emotion engine analyzes the user's latest emotion data and determines the appropriate notification method and content.

[0426] Step 14:

[0427] The server generates a notification message based on the determination result and the selected notification method and sends it to the user.

[0428] Input: Evaluation result data, emotion data, selected notification method.

[0429] Output: A notification message to the user.

[0430] Specific operation: The server converts the judgment result into a soft-spoken sentence and sends it to the user as an email or push notification.

[0431] (Application example 2)

[0432] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0433] Conventional systems can check the vehicle's condition before and after repair using images and use AI to determine the legitimacy of the repair, but they do not take into account the user's emotional state. As a result, users often feel anxious or dissatisfied with the repair process. In particular, similar problems have occurred when determining the quality of products purchased on online shopping sites, causing a decrease in user satisfaction. The objective of this invention is to solve these problems and provide a system that can provide notifications and evaluations that take into account the user's emotions.

[0434] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a pre-repair image taken by the user and saving the image in a database, means for receiving a post-repair image taken by the user after the repair is completed and saving the image in a database, means for comparing the pre-repair and post-repair images using AI and determining the legitimacy of the repair, means for recognizing the user's emotional state and sending the emotional data along with the image to a cloud server, means for checking the user's emotional state before notification and adjusting the notification content based on that, and means for notifying the user of the determination result. This makes it possible to provide a system that takes into consideration not only repair and product quality assessment, but also the user's emotions.

[0435] "User" means an individual or corporation that uses the system to repair and assess the quality of vehicles or products.

[0436] "Image before repair" is image data showing the exterior of a vehicle or product taken by a user before requesting repair.

[0437] "Post-repair image" is image data showing the exterior of a vehicle or product taken after the user has requested repairs.

[0438] The "database" is an information management system that stores information processed and saved by the system, such as image data before and after repair and emotional data.

[0439] A "cloud server" is a remote server that processes, stores, and manages data over the Internet.

[0440] "AI" stands for artificial intelligence, a technology that uses machine learning and deep learning to analyze image data and determine the legitimacy of repairs.

[0441] An "emotion engine" is a software or hardware system for recognizing a user's emotional state and capturing that data.

[0442] "Emotion data" is data that indicates the emotional state of the user as recognized by the emotion engine.

[0443] The "judgment result" is data showing the results of the AI's analysis of pre-repair and post-repair images and its evaluation of the legitimacy of the repair.

[0444] "Notification means" refers to the method, such as email or push notification, that the system uses to notify the user of the judgment results.

[0445] In this invention, a system is implemented in which a user takes a photo of the exterior of a vehicle or product they have purchased or requested repairs on their smartphone, and then sends the image and emotional data to a cloud server. This system uses AI to analyze the images before and after the repair and evaluate the legitimacy of the repair. It also adjusts the notification content based on the user's emotional state and appropriately communicates the results to the user.

[0446] System Configuration

[0447] User terminal

[0448] Users use their smartphones to take pictures of their vehicles or products before and after repairs. The emotion engine recognizes the user's emotional state when taking the pictures and captures that data. This emotional data is then sent to a cloud server along with the images.

[0449] Cloud Server

[0450] The cloud server stores the received image data and emotional data in a database. Furthermore, the AI ​​engine compares the images before and after the repair to evaluate the legitimacy of the repair. Based on the evaluation results and the user's emotional data, the results are communicated to the user via a notification method. In particular, the system evaluates the user's emotional state before the notification and adjusts the notification content accordingly.

[0451] Hardware and software used

[0452] Smartphone: A device that allows users to take pictures, acquire emotion data, and send it to a cloud server.

[0453] OpenCV: A library for capturing images using a smartphone's camera function.

[0454] Emotion Engine: Uses emotion recognition software to analyze the user's emotional state in real time.

[0455] Cloud Server: A remote server that receives and stores data and analyzes it using AI models.

[0456] AI Engine: Artificial intelligence technology that analyzes received images and determines repair and product quality.

[0457] Notification service: A service for notifying users of results via email or push notification.

[0458] Specific examples

[0459] For example, a user orders a new smartphone from an online store and takes a picture of its packaging before receiving it. When the picture is uploaded to a cloud server via a dedicated app, the emotion engine recognizes the user's emotion as "anticipation" and sends the emotion data along with it. Later, when the user takes another picture after receiving the product and uploads it to the cloud server in the same way, the emotion engine recognizes the user's emotion as "excitement."

[0460] The cloud server sends before and after repair images to the AI ​​engine, which judges the quality of the product. As a result, it judges that "the product was delivered without any problems" and notifies the user of the judgment result using an appropriate emotional expression. At this time, the emotion engine takes into account the user's latest emotional state and notifies the user, such as "Don't worry, the product is fine."

[0461] Prompt Sentence Examples

[0462] Prompt: "Check smartphones for damage before and after delivery. Choose a flexible notification method based on user sentiment."

[0463] Model input:

[0464] Previous image: "Previous image data: <image data>"

[0465] After image: "After image data: <image data>"

[0466] Emotion Data: "Emotion Data: <emotional state>"

[0467] Model output:

[0468] Damage check: "No damage."

[0469] Notification text: "Hello, customer. No damage was found before or after delivery of the product. Please rest assured."

[0470] This invention reduces users' concerns about repairs and product quality, and allows them to receive emotionally sensitive service, which is expected to improve user satisfaction.

[0471] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0472] Step 1:

[0473] A user uses a smartphone to take a picture of a product or vehicle before repair. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data. The input is the image data captured by the smartphone camera and the emotion data recognized by the emotion engine, and the output is a set of these data.

[0474] Step 2:

[0475] The device sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format and the emotion data is packaged in JSON format. The input is the captured image data and emotion data, and the output is a transmission request to the cloud server.

[0476] Step 3:

[0477] The cloud server stores the received image data and emotion data in a database. When storing, it generates a unique request ID and manages it together with the image and emotion data. The input is the image data and emotion data sent from the device, and the output is the data stored in the database and the generated request ID.

[0478] Step 4:

[0479] After the repair or receiving of the product, the user uses their smartphone to take a picture of the product after repair or receiving it. At this time, the emotion engine again recognizes the user's emotion and acquires emotion data. The input is the image data captured by the smartphone camera and the recognized emotion data, and the output is a set of these data.

[0480] Step 5:

[0481] The device then sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format again, and the emotion data is packaged in JSON format. The input is the captured image data and emotion data after repair, and the output is a transmission request to the cloud server.

[0482] Step 6:

[0483] The cloud server stores the received image data and emotion data in a database and associates them with the existing request ID. The input is the repaired image data and emotion data sent from the device, and the output is the data stored in the database.

[0484] Step 7:

[0485] The cloud server sends the image data before and after the repair to the AI ​​engine to evaluate the legitimacy of the repair. The input is the image data before and after the repair stored in the database, and the output is the analysis result of the AI ​​engine.

[0486] Step 8:

[0487] The AI ​​engine compares images before and after repair to evaluate the quality and legitimacy of the repair. Specifically, it uses an image analysis algorithm to determine changes in the repair area and the accuracy of the repair. The input is image data before and after the repair, and the output is an evaluation result regarding the legitimacy of the repair.

[0488] Step 9:

[0489] The cloud server receives the evaluation results and stores them in a database. Furthermore, the emotion engine determines the user's emotional state before the notification and adjusts the notification content accordingly. The input is the AI ​​engine's evaluation results and the user's latest emotional data, and the output is the information notified to the user.

[0490] Step 10:

[0491] The cloud server sends the evaluation results to the user via email or push notification via the notification means. The notification content is appropriately adjusted based on the user's emotional state. The input is the notification information sent from the cloud server, and the output is the notification content displayed on the user's device.

[0492] This system makes it possible to provide advanced services that take into consideration the user's emotions during the repair and product quality assessment process.

[0493] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0494] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0495] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0496] [Second embodiment]

[0497] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0498] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0500] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0504] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0507] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0508] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0509] In this invention, a user first takes a photo of the exterior of their vehicle before requesting repairs and uploads the image data to a cloud server. The cloud server then stores the received image data in a database, generates a unique "request ID," and returns it to the user. This request ID is then associated with the post-repair image.

[0510] After the repair is complete, the user takes another photo of the vehicle's exterior and uploads the image data to the cloud server. The cloud server receives the post-repair image data, associates it with the request ID generated earlier, and stores it in a database.

[0511] Next, the cloud server uses AI to compare the before and after images. The AI ​​analyzes the images and evaluates the changes in the repaired area and the quality of the repair. For example, it checks whether a dent in the door before the repair is gone after the repair, and whether the paint is applied properly.

[0512] After the pass / fail result is obtained, the cloud server records the result in a database and notifies the user via email or push notification on a dedicated app.

[0513] Program processing description

[0514] 1. The user takes a photo of the car's exterior using a smartphone.

[0515] 2. The device (user's smartphone) sends an image upload request to the cloud server from a dedicated app.

[0516] 3. The server saves the received image data, generates a unique request ID, and returns it to the user.

[0517] Next, after the repair is completed, the procedure will proceed as follows:

[0518] 1. The user takes another photo of the exterior of the vehicle after repairs.

[0519] 2. The device sends a request to upload images of the repaired item to the cloud server.

[0520] 3. The server stores the received repaired images in a database and associates them with the request ID.

[0521] Next, a comparison of the before and after images begins.

[0522] 1. The server sends image data before and after repair to the AI ​​engine.

[0523] 2. AI performs image analysis to evaluate changes in the repair area and the quality of the repair.

[0524] Finally, a judgment result is generated and notified to the user.

[0525] 1. The server receives the AI ​​analysis results and stores them in a database.

[0526] 2. The server notifies the user of the result of the judgment, for example, "the repair is valid" or "the repair is incorrect."

[0527] 3. The user receives a notification via a dedicated app or email and confirms that the repair is legitimate.

[0528] Specific examples

[0529] Uploading images before requesting repair

[0530] The user takes a photo of the left side of the car using a smartphone.

[0531] The device uploads images to the cloud using a dedicated app.

[0532] The server receives the image and stores it in the database with "Request ID 12345." The Request ID is returned to the user.

[0533] Uploading images after repair is complete

[0534] The user re-photographs the left side of the repaired car.

[0535] The device uploads images of the repaired item to the cloud.

[0536] The server receives the image and stores it, linking it to request ID 12345.

[0537] Image judgment by AI

[0538] The server sends before and after images of the repair for request ID 12345 to the AI ​​engine.

[0539] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[0540] Notification of the results

[0541] The server stores the determination result that "the repair is legitimate" in a database.

[0542] The server will notify the user of the results via email or in-app push notification.

[0543] The user receives a notification and checks the details in the app.

[0544] The above is a specific embodiment of this system.

[0545] The processing flow will be explained below.

[0546] Step 1:

[0547] Before requesting repairs, the user takes a photo of the vehicle's exterior using the camera on their smartphone.

[0548] Step 2:

[0549] The device (user's smartphone) launches a dedicated app and sends a request to upload the captured images to a cloud server.

[0550] Step 3:

[0551] The server receives the uploaded image data and stores it in a database. At this time, it generates a new unique "request ID" and returns it to the user. The request ID is used to associate the before and after images of the repair.

[0552] Step 4:

[0553] After the repairs are completed, the user again uses the smartphone to take a photo of the exterior of the vehicle after repairs.

[0554] Step 5:

[0555] The device sends a request to upload the repaired image to a cloud server via a dedicated app.

[0556] Step 6:

[0557] The server receives the repaired image data and stores it in the database, linking it to the request ID generated earlier.

[0558] Step 7:

[0559] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine uses image analysis technology to evaluate the changes in the repaired area and the quality of the repair. For example, it checks whether dents have been repaired and whether the paint has been applied properly.

[0560] Step 8:

[0561] The AI ​​returns the analysis results and determines whether the repair is legitimate or illegitimate, outputting the results as a clear status such as "legitimate" or "incorrect."

[0562] Step 9:

[0563] The server stores the judgment results received from the AI ​​in a database. These judgment results are assigned a request ID.

[0564] Step 10:

[0565] The server then begins the process of notifying the user of the result of the judgment, specifically by sending an email or a push notification to a dedicated app.

[0566] Step 11:

[0567] Users receive notifications and can view detailed results within the app, allowing them to objectively determine whether the repair was legitimate.

[0568] Example 1

[0569] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0570] There is a need for a system that allows users to easily check whether their car repairs have been performed properly. Conventional methods require direct visual inspection of the condition before and after repairs, which takes time and effort. In addition, users have no way to objectively evaluate the quality of the repairs, so they may not notice if the repairs have been performed improperly. A system that solves these problems is needed.

[0571] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0572] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in storage, means for receiving post-repair images taken by the user after the repair is completed and saving the images in storage, means for analyzing the pre-repair and post-repair images using a generative AI model to determine the legitimacy of the repair, and means for notifying the user of the determination result, thereby enabling the user to quickly and accurately confirm whether the car repair was performed appropriately.

[0573] A "pre-repair image" is digital image data of the exterior of a vehicle taken by a user before repairs are made.

[0574] "Post-repair image" is digital image data of the exterior of the vehicle taken by the user after repairs have been completed.

[0575] "Storage" refers to storage devices and cloud services for storing digital data.

[0576] A "generative AI model" is an artificial intelligence model trained using machine learning techniques and used to perform a specific task, such as image analysis or judgment.

[0577] A "server" is a computer system that receives requests over a network, stores and processes data, and communicates with other computers.

[0578] "User" refers to an individual or company that uses this system to request automobile repairs.

[0579] "Analysis" is the process of using AI models to detect, compare, and evaluate the content of digital images.

[0580] The "judgment result" is information regarding the legitimacy of the repair obtained as a result of analysis by the generative AI model.

[0581] "Notification" refers to information sent from the system to the user, including emails and app push notifications.

[0582] The system of the present invention allows users to upload images taken before and after requesting a car repair to a cloud server, and then uses a generative AI model to determine the legitimacy of the repair. A specific embodiment of this system is described below.

[0583] First, the user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then selects the captured image using a dedicated app (e.g., CarRepairUploader) and presses the upload button. The device then sends this image data to a cloud server via the Internet. The cloud server receives the image data and stores it in storage (e.g., an Amazon S3 bucket). It then generates a unique "request ID" (e.g., E12345) and returns it to the user.

[0584] Next, after the repair is complete, the user takes a photo of the vehicle's exterior after repairs using the same procedure. The user then uses the dedicated app again to select the post-repair image and press the upload button. The device then sends the post-repair image data to the cloud server. The cloud server again receives the image data, associates it with the request ID (e.g., E12345) generated earlier, and stores it in storage.

[0585] The cloud server sends the image data before and after repair to a generative AI model (e.g., Amazon Rekognition). The generative AI model analyzes the images and evaluates changes in the repair area and the quality of the repair. After obtaining a judgment result, the cloud server stores the result in a database (e.g., Amazon RDS) and notifies the user. Notifications are sent via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging). Users receive a notification and can view detailed information in a dedicated app.

[0586] Specific examples

[0587] As a concrete example, the image upload before requesting repair is shown below.

[0588] Uploading images before requesting repair

[0589] The user takes a photo of the left side of the car using a smartphone (iPhone 13 Pro).

[0590] The device (iPhone 13 Pro) uploads images to the cloud server using a dedicated app (CarRepairUploader).

[0591] The server receives the image and stores it in storage (Amazon S3 bucket) with "Request ID E12345." The Request ID is returned to the user.

[0592] Uploading images after repair is complete

[0593] The user takes another photograph of the left side of the car after repairs.

[0594] The device (iPhone 13 Pro) also uploads images of the repaired item to a cloud server using a dedicated app (CarRepairUploader).

[0595] The server receives the image, associates it with the request ID E12345, and stores it in storage.

[0596] Image judgment by AI

[0597] The server sends the before and after images of the repair for request ID E12345 to the generated AI model (Amazon Rekognition).

[0598] The AI ​​engine analyzes the images and checks for changes in the repaired area, for example, to see if a dent in the door before the repair has disappeared after the repair.

[0599] Notification of the results

[0600] The server stores the determination that the repair is valid in a database (Amazon RDS).

[0601] The server notifies the user of the result, for example via email (SendGrid) or in-app push notification (Firebase Cloud Messaging).

[0602] The user receives a notification and checks the details using a dedicated app (CarRepairUploader).

[0603] Using generative AI models, the validity of vehicle repairs can be efficiently and accurately assessed. This system provides users with a means to objectively verify the quality of repairs and can prevent inappropriate repairs.

[0604] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0605] Step 1:

[0606] The user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then opens a dedicated app (e.g., CarRepairUploader), selects the image, and presses the upload button.

[0607] Input: High-resolution image data before repair taken with a smartphone

[0608] How it works: The user selects an image in the app and taps the upload button.

[0609] Output: The captured image data is saved on the device and an upload request is sent to the server.

[0610] Step 2:

[0611] The device sends the captured image data to a cloud server via the Internet.

[0612] Input: Image data before repair and request to upload to server

[0613] How it works: Your device uses its internet connection to send captured images to a cloud server.

[0614] Output: The image data arrives at the cloud server.

[0615] Step 3:

[0616] The server stores the received image data in storage (e.g., Amazon S3 bucket) and generates a unique "request ID" (e.g., E12345). The server returns the generated request ID to the user.

[0617] Input: Image data before repair submitted

[0618] Operation: Image data is stored in Amazon S3, a unique ID generation algorithm is used to generate a request ID, and the generated request ID is returned to the device.

[0619] Output: Returns the image data stored in the storage and the request ID.

[0620] Step 4:

[0621] After the repair is complete, the user takes another photo of the vehicle's exterior with their smartphone, selects the image after repair using the dedicated app, and presses the upload button.

[0622] Input: High-resolution image data of the repaired item taken with a smartphone

[0623] How it works: The user selects an image in the app and taps the upload button.

[0624] Output: The captured image data is saved on the device and an upload request is sent to the server.

[0625] Step 5:

[0626] The device then sends the repaired image data back to the cloud server via the Internet.

[0627] Input: Repaired image data and upload request to the server

[0628] How it works: Your device uses its internet connection to send captured images to a cloud server.

[0629] Output: The image data arrives at the cloud server.

[0630] Step 6:

[0631] The server stores the received repaired image data in storage (e.g., Amazon S3 bucket) and associates it with the request ID (e.g., E12345) generated earlier.

[0632] Input: The image data after repair and the existing request ID

[0633] Operation: The repaired image data is saved in Amazon S3 and linked to the existing request ID.

[0634] Output: Image data stored in storage and ID link information

[0635] Step 7:

[0636] The server sends the image data before and after the repair to a generative AI model (e.g., Amazon Rekognition), which analyzes the images and evaluates the changes in the repaired area and the quality of the repair.

[0637] Input: Before and after image data

[0638] How it works: The server sends image data to the generative AI model, which then analyzes the image.

[0639] Output: Evaluation results regarding changes in repaired areas and repair quality

[0640] Step 8:

[0641] The server receives the analysis results from the AI ​​engine, stores them in a database (e.g., Amazon RDS), and notifies the user of the results via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging).

[0642] Input: Analysis results from a generative AI model

[0643] Behavior: The analysis results are inserted into a database and the results are notified to the user using an email sending API or push notification API.

[0644] Output: Analysis results stored in a database and notifications to the user

[0645] Step 9:

[0646] Users will receive a notification and can check detailed information in a dedicated app.

[0647] Input: Notification of judgment results by email or push notification

[0648] How it works: The user opens the app and taps the "View Repair Results" button to view the results.

[0649] Output: Repair assessment results displayed in the user's app

[0650] The above is the flow of the program processing of the present invention.

[0651] (Application example 1)

[0652] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0653] Conventional technology requires users to manually take photos of the vehicle's exterior and upload the images before and after repair to a cloud server, which can lead to manual labor and errors. Furthermore, there is a lack of a means to objectively evaluate the legitimacy of repairs, making reliable damage diagnosis difficult. This makes it difficult to control the quality of repairs and achieve efficient maintenance.

[0654] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0655] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, means for notifying the user of the determination result, means for automatically photographing the vehicle exterior using an on-board camera, means for uploading the image data to a cloud server, means for the cloud server to generate a unique request ID, and means for saving the photographed images in a database based on the generated request ID. This enables automatic photographing of the vehicle exterior and uploading the images to the cloud server, allowing AI to determine whether there is damage, enabling efficient and reliable damage diagnosis.

[0656] "User" refers to a person who uses the system to request repairs or inspections of a vehicle, or the owner of such a vehicle.

[0657] The "image before repair" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device before the vehicle is repaired.

[0658] A "database" is a system for accumulating, storing, and managing received image data and related information.

[0659] "Post-repair image" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device after the vehicle has been repaired.

[0660] "AI" refers to artificial intelligence technology, a general term for programs and algorithms that have the ability to analyze image data and compare before and after repairs.

[0661] A "cloud server" is a physical or virtual server used over a network that provides resources for storing and processing image data and related information.

[0662] A "request ID" is a unique identification code generated by the cloud server that is linked to a specific repair request.

[0663] "Photography device" refers to a camera or other photographic equipment that is mounted on a vehicle and automatically photographs the exterior of the vehicle.

[0664] The "judgment result" indicates the result of image analysis by AI, an evaluation of the validity of the repair and the presence or absence of damage, and its conclusion.

[0665] The "notification means" is a function for notifying the user of the judgment result, and includes email, push notification, etc.

[0666] This invention relates to a system that automatically diagnoses damage to autonomous vehicles. It eliminates the need for users to manually upload images of the vehicle before and after repairs, and also enables automatic evaluation of the quality of repairs by performing image analysis.

[0667] The system consists of the following main components:

[0668] 1. Vehicle-mounted imaging equipment:

[0669] The camera system installed in the vehicle automatically captures the exterior of the vehicle, mainly before the vehicle departs and after it returns, eliminating the need for the user to manually take photos.

[0670] 2. Cloud Server:

[0671] The captured image data is uploaded to a cloud server, which stores the received image data in a database and generates a unique request ID. The generated request ID is linked to the images before and after repair.

[0672] 3. AI image analysis engine:

[0673] AI analyzes the image data before and after repair stored on a cloud server. This AI uses image analysis technology to evaluate changes in the repaired area and the quality of the repair. Specifically, it checks whether dents and scratches that existed before the repair have been properly repaired and whether the paint has been applied accurately.

[0674] 4. Notification system:

[0675] The AI's assessment results are stored on a cloud server and notified to users and fleet managers. This notification is mainly via email or push notification, allowing users to immediately confirm the legitimacy of repairs.

[0676] Hardware and software used

[0677] Camera system: Used to capture the exterior of the vehicle. High-resolution cameras are recommended.

[0678] Cloud server: Common cloud servers such as AWS and Google Cloud are used.

[0679] Database system: A relational database such as MySQL or PostgreSQL is used.

[0680] AI image analysis software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[0681] Specific examples

[0682] A specific example of how this system can be used is shown below.

[0683] Photograph and upload before requesting repair:

[0684] The vehicle automatically takes a photo of the exterior before departing.

[0685] The vehicle system uploads the image data to a cloud server.

[0686] The cloud server receives the image data and stores it in the database as request ID 12345.

[0687] Photograph and upload after repair is complete:

[0688] After the vehicle returns from repairs, the exterior is automatically photographed again.

[0689] The vehicle system uploads post-repair image data to a cloud server.

[0690] The cloud server receives the image data and stores it, linking it to request ID 12345.

[0691] AI image assessment:

[0692] The cloud server sends image data before and after repair for request ID 12345 to the AI ​​engine.

[0693] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[0694] Notification of decision:

[0695] The cloud server stores the determination result that "the repair is legitimate" in a database.

[0696] The cloud server notifies the user of the results via email or in-app push notification.

[0697] The user receives a notification and checks the details in the app.

[0698] Prompt Sentence Examples

[0699] Develop an application that allows users to take photos of the exterior of autonomous vehicles using a smartphone and upload the images to the cloud. The application should have the ability to use AI to compare before and after images to determine the quality of the repair.

[0700] The above is a specific embodiment for carrying out the present invention.

[0701] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0702] Step 1:

[0703] A camera mounted on the vehicle automatically photographs the vehicle's exterior. The images are captured by a high-resolution camera, and metadata such as location and timestamp are also saved. (Input): Vehicle exterior. (Output): High-resolution image.

[0704] Step 2:

[0705] The device uploads the captured image data to the cloud server. At this time, the vehicle ID and other identification information are also sent. (Input): Captured image data and related identification information, (Output): Image data uploaded to the cloud server.

[0706] Step 3:

[0707] The cloud server stores the received image data in a database and generates a unique request ID, which makes it easier to identify images before and after repair. (Input): Received image data and identification information, (Output): Image data stored in the database and generated request ID.

[0708] Step 4:

[0709] After the user requests repairs, the vehicle automatically photographs its exterior once the repairs are complete. The photographic device captures the exterior of the vehicle after repairs and generates image data again. (Input): Appearance of the repaired vehicle, (Output): High-resolution image after repairs.

[0710] Step 5:

[0711] The terminal re-uploads the repaired image data to the cloud server. At this time, identification information including the vehicle ID and request ID is also sent. (Input): Repaired image data and related identification information, (Output): Upload of repaired image data to the cloud server.

[0712] Step 6:

[0713] The cloud server receives the image data after repair and stores it in a database, linking it to the request ID already saved. (Input): Received image data after repair and request ID, (Output): Image data after repair saved in the database.

[0714] Step 7:

[0715] The cloud server sends the image data before and after repair to an AI image analysis engine. This AI engine performs image analysis using generative AI models such as TensorFlow and PyTorch. (Input): Image data before and after repair. (Output): Analysis results regarding the quality of the repair.

[0716] Step 8:

[0717] The AI ​​image analysis engine compares image data before and after repair, assessing changes in each part and the quality of the repair. It determines whether the changes are appropriate and concludes that the repair is appropriate or inaccurate. (Input): Image data before and after repair, (Output): Evaluation results regarding the quality of the repair.

[0718] Step 9:

[0719] The server stores the AI ​​analysis results in a database and notifies the user. Notifications are mainly sent via email or push notifications. (Input): AI analysis results, (Output): Notification to the user.

[0720] Step 10:

[0721] The user receives a notification and can check the detailed repair quality results in the app, which allows them to know whether the repair was justified or inappropriate. (Input): Notification from the server, (Output): Display of analysis results in the user's app.

[0722] This series of processes not only allows users to easily and accurately grasp the condition of their vehicle before and after repairs, but also allows them to use the data to evaluate the quality of repairers.

[0723] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0724] The present invention provides a system that combines a process in which a user takes exterior images of a vehicle before and after requesting repairs and uploads them to a cloud server with an emotion engine that recognizes the user's emotions.

[0725] System Configuration

[0726] 1. The user uses their smartphone to take a photo of the car's exterior before repairs and uploads the image to a cloud server via a dedicated app. At this point, the emotion engine recognizes the user's emotional state and sends that data along with the image.

[0727] 2. The device (user's smartphone) generates a request to send the captured image and the user's emotional data to the cloud server and sends it.

[0728] 3. The server stores the received image data and emotion data in a database, generates a unique "request ID," and returns it to the user. The request ID is used to associate the images and emotion data before and after repair.

[0729] The procedures after the repair is completed are almost the same as those described above.

[0730] 4. After the repair is complete, the user again uses their smartphone to take a photo of the vehicle's exterior and uploads the image to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state and sends that data along with the image.

[0731] 5. The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[0732] 6. The server receives the repaired image data and emotion data, associates them with the request ID, and stores them in a database.

[0733] 7. The server uses AI to compare before and after images and evaluate the changes and quality of the repair, such as whether the dent in the door was fixed or whether the paint was applied properly.

[0734] 8. The AI ​​generates the results of the image analysis and returns them to the server, which stores them in a database.

[0735] 9. The server then initiates the procedure to notify the user of the result of the judgment. At this time, the emotion engine checks the user's emotional state before the notification and selects an appropriate notification method based on the results. For example, if the user is in an unstable emotional state, the content of the notification can be flexibly adjusted.

[0736] Specific examples

[0737] Uploading images before requesting repair

[0738] The user takes a photo of the left side of the car using their smartphone and uploads the image to a cloud server via a dedicated app. The emotion engine recognizes the user's emotions and sends emotional data such as "relief" along with the image.

[0739] The device sends a request to the cloud server. The server receives the image and emotion data and stores it in the database under "Request ID 12345."

[0740] Uploading images after repair is complete

[0741] The user then takes another photo of the left side of the repaired car and uploads it to the cloud server via a dedicated app. The emotion engine then recognizes the user's emotions again, and includes emotional data such as "expectation."

[0742] The device sends a request to the cloud server. The server receives the image and emotion data, associates it with request ID 12345, and stores it.

[0743] AI-based image judgment and emotional response

[0744] The server sends the image data before and after the repair to the AI ​​engine for analysis. The AI ​​evaluates the quality of the repair and generates a judgment that the repair is legitimate.

[0745] The server receives the judgment result and stores it in a database. The emotion engine checks the user's latest emotional state and selects an appropriate notification method (e.g., a soft-spoken notification message).

[0746] The server sends the result of the assessment to the user via email or push notification, which includes the assessment result and additional reassurance information according to the user's emotional state.

[0747] The system allows users to ensure the validity of repairs and also provides emotional consideration during the process.

[0748] The processing flow will be explained below.

[0749] Step 1:

[0750] The user uses a smartphone to take a photo of the car's exterior before repairs are made, and the image is then uploaded to a cloud server using a dedicated app.

[0751] Step 2:

[0752] The device generates and sends a request to upload the pre-repair image to a cloud server via a dedicated app. It also analyzes the user's emotional state (e.g., "relief" or "anxiety") through an emotion engine and sends that data together.

[0753] Step 3:

[0754] The server receives the uploaded image of the item before repair and the user's emotion data, stores them in a database, and generates a new "request ID" and returns it to the user.

[0755] Step 4:

[0756] After the repair is complete, the user takes another photo of the exterior of the car using their smartphone.

[0757] Step 5:

[0758] The device generates and sends a request to upload the repaired image and the user's emotion data (analyzed by the emotion engine) to the cloud server.

[0759] Step 6:

[0760] The server receives the post-repair image and emotion data and stores them in the database with the same request ID as the pre-repair image.

[0761] Step 7:

[0762] The server sends the image data before and after the repair to the AI ​​engine and requests image analysis, which then begins the process of evaluating changes in the repaired area and the quality of the repair.

[0763] Step 8:

[0764] The AI ​​engine analyzes images to determine whether repairs are warranted, for example, checking that a dent in a door has been repaired or that the paint is properly applied.

[0765] Step 9:

[0766] The AI ​​engine generates a verdict and returns the result to the server, for example, "The repair is valid" or "The repair is incorrect."

[0767] Step 10:

[0768] The server stores the received judgment results in a database. At this time, the emotion engine checks the user's latest emotional state and determines how to notify the judgment results based on that result.

[0769] Step 11:

[0770] The server then initiates a procedure to notify the user of the result of the assessment. For example, if the user is in an anxious emotional state, the server may soften the content of the notification.

[0771] Step 12:

[0772] The user receives the results via a dedicated app or email, which includes additional reassurance information tailored to the user's emotional state.

[0773] Specific examples

[0774] Uploading images before requesting repair

[0775] The user takes a photo of the left side of the car with their smartphone. The app prepares the image for upload to a cloud server. The emotion engine analyzes the user's emotion as "relief" and sends the image to the server.

[0776] The device (smartphone) sends images and emotional data to a cloud server.

[0777] Uploading repaired images

[0778] The user takes another photo of the left side of the car after the repair is complete. The app prepares to upload the repaired image to the cloud server. The emotion engine analyzes the user's emotion as "expectation" and sends it to the server.

[0779] The device (smartphone) sends images of the repaired item and emotional data to a cloud server.

[0780] AI-based image judgment and emotional response

[0781] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine performs the image analysis and generates a result that the repair is legitimate.

[0782] The server stores the result in a database and uses the emotion engine to check the user's current emotional state, which determines that the user needs to be notified in a softer tone.

[0783] The server sends the result of the assessment via email or push notification to a dedicated app, which includes additional information to reassure the user along with the result.

[0784] Users receive a notification and can check the detailed results in a dedicated app. The notification is designed with emotional consideration, giving them peace of mind regarding the repair details.

[0785] The above is a specific embodiment of the invention in which an emotion engine is combined.

[0786] Example 2

[0787] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0788] While conventional car repair evaluation systems can judge the legitimacy of repairs by comparing before and after images, they lack a notification method that takes into account the emotional state of the repair requester, which has led to issues with not being able to fully achieve the requester's satisfaction or sense of security. Furthermore, conventional systems do not use emotional data for evaluation, and so the evaluation of repair companies remains one-sided.

[0789] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0790] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for recognizing the user's emotional state and transmitting emotional data along with the images, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, and means for selecting an appropriate notification method and notifying the user based on the determination result and the user's latest emotional state. This not only makes it possible to objectively evaluate the legitimacy of the repair, but also increases user satisfaction and peace of mind by adopting a notification method that takes the user's emotional state into consideration.

[0791] "User" refers to any individual or corporation that uses this system.

[0792] A "terminal" is a device used by a user, such as a smartphone or tablet, that is primarily used for taking photos and transmitting data.

[0793] "Server" refers to a central processing unit that manages and stores received data and performs AI analysis and notification processing.

[0794] A "cloud server" is a server used over the Internet that includes resources for storing, processing, and analyzing data.

[0795] A "database" refers to a system that centrally manages information and allows it to be stored, searched, and updated.

[0796] An "emotion engine" refers to software or algorithms that recognize a user's emotional state and process it as data.

[0797] "Image data" refers to images of the vehicle before and after repair that are taken by the user and uploaded to the cloud server.

[0798] "Emotion data" refers to data that indicates the user's emotional state as recognized by the emotion engine.

[0799] "Request ID" refers to a unique identifier generated to identify and manage a series of data related to a repair request.

[0800] "AI engine" refers to the artificial intelligence algorithm used to analyze before and after repair images and evaluate the quality of the repair.

[0801] "Judgment result" refers to the evaluation result obtained by the AI ​​engine by comparing and analyzing images before and after repair.

[0802] "Notification means" refers to a function that sends information to the user in an appropriate manner based on the judgment result and emotion data.

[0803] A "transaction" refers to a mechanism for processing a series of operations performed within a database as a single unit and maintaining its consistency.

[0804] "HTTPS protocol" refers to an internet protocol for secure communications.

[0805] MODE FOR CARRYING OUT THE INVENTION

[0806] This invention is a system that records the process in detail when a user requests car repairs, and evaluates the quality of the repair based on images taken before and after the repair and the user's emotional state. This system is composed of the user's device, a cloud server, a database, an AI engine, and an emotion engine.

[0807] Hardware and software used

[0808] Device: A smartphone or tablet operated by the user

[0809] Cloud server: A server that stores, processes, and analyzes data (e.g., AWS, Azure)

[0810] Database: A system for storing image data and emotion data (e.g., MySQL, PostgreSQL)

[0811] Emotion engine: Software for recognizing user emotions (e.g., Emotion API, IBM Watson Tone Analyzer)

[0812] AI engine: Algorithm that analyzes before and after repair images and evaluates the quality of the repair (e.g., TensorFlow, PyTorch)

[0813] System details

[0814] Before requesting car repairs, a user takes a photo of the vehicle's condition using their smartphone and uploads the image to a cloud server using a dedicated app. At this time, the emotion engine recognizes the user's emotional state (e.g., "relief") and sends that data along with the image.

[0815] The device generates a request to send the captured image data and the user's emotion data to the cloud server, and sends it to the cloud server using the HTTPS protocol.

[0816] The server stores the received images and emotion data in a database and generates a unique request ID, which is used to associate the images and emotion data before and after repair.

[0817] After the repair is complete, the user takes another photo of the vehicle's condition with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state (e.g., "expectation") and sends that data along with the image.

[0818] The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[0819] The server receives the post-repair images and emotion data, associates them with the request ID, and stores them in a database. It then sends the pre- and post-repair image data to the AI ​​engine, requesting it to evaluate the quality of the repair.

[0820] The AI ​​engine compares image data before and after the repair, assesses changes in the repaired area, and evaluates the quality of the repair. The results are sent back to the server and stored in a database.

[0821] The emotion engine checks the user's latest emotional state before notifying them of the evaluation results and selects an appropriate notification method (e.g., soft-spoken text).

[0822] The server generates a message to notify the user based on the judgment result and emotion data, and sends it via email or push notification.

[0823] Specific examples

[0824] For example, if a user takes a photo of the left side of a car using a smartphone and uploads the image to a cloud server using a dedicated app, the emotion engine will recognize the user's emotion as "relief" and send that data along with the image.

[0825] The device sends a request to the cloud server, and the server receives the image and emotion data and stores it in the database with request ID "12345."

[0826] After the repair is complete, the user takes another photo of the left side of the vehicle with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotion as "expectation" and sends the data along with it.

[0827] The device sends a request to the cloud server, and the server receives the image and emotion data, associates it with the request ID "12345," and stores it.

[0828] The server sends the image data before and after the repair to the AI ​​engine, which evaluates the quality of the repair and generates a result that the repair is valid. The server receives the result, and the emotion engine checks the user's latest emotional state and creates a soft-spoken notification message.

[0829] The server will send an email or push notification stating that the repair is legitimate, along with information that will reassure the user.

[0830] Prompt Sentence Examples

[0831] "The user takes a photo of the vehicle before repair and uploads it to the cloud server via a dedicated app. At this time, the emotion engine recognizes the user's emotional state and sends data indicating a sense of security."

[0832] The cloud server receives the image and emotion data, generates a request ID, and returns it to the user.

[0833] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0834] Step 1:

[0835] A user uses a smartphone to take a picture of the car before repair.

[0836] Input: Use the camera function of your smartphone to take a photo of the current state of the car.

[0837] Output: Vehicle exterior image data (e.g. JPEG file).

[0838] Specific operation: The user launches the app and uses the camera function to take a photo of the area to be repaired, such as the left side of the vehicle.

[0839] Step 2:

[0840] The terminal acquires image data and the user's emotional state.

[0841] Input: Image data of the vehicle and the user's facial expression captured in real time by the camera.

[0842] Output: Vehicle image data, emotion data (e.g., relief, expectation).

[0843] Specific operation: The device's emotion engine recognizes the user's face and analyzes emotions from facial expression data. For example, it uses the Emotion API to measure feelings of relief.

[0844] Step 3:

[0845] The terminal generates and transmits a request for transmitting image data and emotion data to a cloud server.

[0846] Input: Vehicle image data, emotion data, user ID, timestamp.

[0847] Output: HTTPS outgoing request to cloud server.

[0848] Specific operation: The device generates an HTTPS request and sends it to the API endpoint of the cloud server, including image data and emotion data.

[0849] Step 4:

[0850] The server stores the received image data and emotion data in a database and generates a unique request ID.

[0851] Input: Image data, emotion data, user ID, and timestamp sent to the cloud server.

[0852] Output: The database entry and the generated request ID.

[0853] Specific operation: The server starts a transaction, records the received data in the database, and simultaneously generates a request ID and returns it to the user.

[0854] Step 5:

[0855] The server returns the request ID to the user.

[0856] Input: Generated request ID.

[0857] Output: Request ID notification to the user's device.

[0858] Specific behavior: The server generates a response containing the request ID and sends it back to the user's device via HTTPS. A notification is displayed in the user's app.

[0859] Step 6:

[0860] After the repair is completed, the user takes another picture of the repaired vehicle using their smartphone.

[0861] Input: What the vehicle will look like after repairs are complete.

[0862] Output: Image data of the vehicle's exterior after repair (e.g., JPEG file).

[0863] Specific operation: The user again uses the camera function to take a picture of the condition of the vehicle after repairs.

[0864] Step 7:

[0865] The terminal acquires the image data after repair and the user's emotional state again.

[0866] Input: A captured image of the vehicle after repair, and a user's facial expression capture.

[0867] Output: Repaired image data, emotion data (e.g., expectation).

[0868] Specific operation: The device's emotion engine performs facial recognition and analyzes and generates emotion data again.

[0869] Step 8:

[0870] The terminal generates and transmits a request for transmitting the post-repair image and emotion data to a cloud server.

[0871] Input: Repaired image data, emotion data, user ID, request ID, timestamp.

[0872] Output: HTTPS outgoing request to cloud server.

[0873] Specific operation: The device generates another HTTPS request and sends the image and emotion data to the cloud server.

[0874] Step 9:

[0875] The server receives the repaired image data and emotion data, links them to the request ID, and stores them in a database.

[0876] Input: Repaired image data, emotion data, and request ID sent to the cloud server.

[0877] Output: Updated database entry.

[0878] Specific operation: The server associates the received data based on the request ID and stores it in the database. Data integrity is ensured using transactions.

[0879] Step 10:

[0880] The server sends the image data before and after the repair to the AI ​​engine and asks it to evaluate the quality of the repair.

[0881] Input: Image data before and after repair.

[0882] Output: Data analysis request to the AI ​​engine.

[0883] Specific operation: The server accesses the AI ​​engine's API, sends image data before and after repair, and requests an evaluation.

[0884] Step 11:

[0885] The AI ​​engine analyzes the image data and evaluates changes in the repair area and the quality of the repair.

[0886] Input: Image data before and after repair.

[0887] Output: Evaluation result data (e.g., "Repair is justified").

[0888] How it works: The AI ​​engine applies image analysis algorithms to evaluate changes between images to determine the quality of the repair, and generates an assessment result that is sent back to the server.

[0889] Step 12:

[0890] The server receives the evaluation results from the AI ​​engine and stores them in a database.

[0891] Input: Evaluation result data sent from the AI ​​engine.

[0892] Output: Evaluation results stored in a database.

[0893] Specific operation: The received evaluation results are associated with the request ID and stored in the database.

[0894] Step 13:

[0895] The emotion engine checks the user's latest emotional state and selects the appropriate notification method.

[0896] Input: The user's latest emotion data.

[0897] Output: Select an appropriate notification method (e.g., a soft-spoken notification).

[0898] Specific operation: The emotion engine analyzes the user's latest emotion data and determines the appropriate notification method and content.

[0899] Step 14:

[0900] The server generates a notification message based on the determination result and the selected notification method and sends it to the user.

[0901] Input: Evaluation result data, emotion data, selected notification method.

[0902] Output: A notification message to the user.

[0903] Specific operation: The server converts the judgment result into a soft-spoken sentence and sends it to the user as an email or push notification.

[0904] (Application example 2)

[0905] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0906] Conventional systems can check the vehicle's condition before and after repair using images and use AI to determine the legitimacy of the repair, but they do not take into account the user's emotional state. As a result, users often feel anxious or dissatisfied with the repair process. In particular, similar problems have occurred when determining the quality of products purchased on online shopping sites, causing a decrease in user satisfaction. The objective of this invention is to solve these problems and provide a system that can provide notifications and evaluations that take into account the user's emotions.

[0907] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a pre-repair image taken by the user and saving the image in a database, means for receiving a post-repair image taken by the user after the repair is completed and saving the image in a database, means for comparing the pre-repair and post-repair images using AI and determining the legitimacy of the repair, means for recognizing the user's emotional state and sending the emotional data along with the image to a cloud server, means for checking the user's emotional state before notification and adjusting the notification content based on that, and means for notifying the user of the determination result. This makes it possible to provide a system that takes into consideration not only repair and product quality assessment, but also the user's emotions.

[0908] "User" means an individual or corporation that uses the system to repair and assess the quality of vehicles or products.

[0909] "Image before repair" is image data showing the exterior of a vehicle or product taken by a user before requesting repair.

[0910] "Post-repair image" is image data showing the exterior of a vehicle or product taken after the user has requested repairs.

[0911] The "database" is an information management system that stores information processed and saved by the system, such as image data before and after repair and emotional data.

[0912] A "cloud server" is a remote server that processes, stores, and manages data over the Internet.

[0913] "AI" stands for artificial intelligence, a technology that uses machine learning and deep learning to analyze image data and determine the legitimacy of repairs.

[0914] An "emotion engine" is a software or hardware system for recognizing a user's emotional state and capturing that data.

[0915] "Emotion data" is data that indicates the emotional state of the user as recognized by the emotion engine.

[0916] The "judgment result" is data showing the results of the AI's analysis of pre-repair and post-repair images and its evaluation of the legitimacy of the repair.

[0917] "Notification means" refers to the method, such as email or push notification, that the system uses to notify the user of the judgment results.

[0918] In this invention, a system is implemented in which a user takes a photo of the exterior of a vehicle or product they have purchased or requested repairs on their smartphone, and then sends the image and emotional data to a cloud server. This system uses AI to analyze the images before and after the repair and evaluate the legitimacy of the repair. It also adjusts the notification content based on the user's emotional state and appropriately communicates the results to the user.

[0919] System Configuration

[0920] User terminal

[0921] Users use their smartphones to take pictures of their vehicles or products before and after repairs. The emotion engine recognizes the user's emotional state when taking the pictures and captures that data. This emotional data is then sent to a cloud server along with the images.

[0922] Cloud Server

[0923] The cloud server stores the received image data and emotional data in a database. Furthermore, the AI ​​engine compares the images before and after the repair to evaluate the legitimacy of the repair. Based on the evaluation results and the user's emotional data, the results are communicated to the user via a notification method. In particular, the system evaluates the user's emotional state before the notification and adjusts the notification content accordingly.

[0924] Hardware and software used

[0925] Smartphone: A device that allows users to take pictures, acquire emotion data, and send it to a cloud server.

[0926] OpenCV: A library for capturing images using a smartphone's camera function.

[0927] Emotion Engine: Uses emotion recognition software to analyze the user's emotional state in real time.

[0928] Cloud Server: A remote server that receives and stores data and analyzes it using AI models.

[0929] AI Engine: Artificial intelligence technology that analyzes received images and determines repair and product quality.

[0930] Notification service: A service for notifying users of results via email or push notification.

[0931] Specific examples

[0932] For example, a user orders a new smartphone from an online store and takes a picture of its packaging before receiving it. When the picture is uploaded to a cloud server via a dedicated app, the emotion engine recognizes the user's emotion as "anticipation" and sends the emotion data along with it. Later, when the user takes another picture after receiving the product and uploads it to the cloud server in the same way, the emotion engine recognizes the user's emotion as "excitement."

[0933] The cloud server sends before and after repair images to the AI ​​engine, which judges the quality of the product. As a result, it judges that "the product was delivered without any problems" and notifies the user of the judgment result using an appropriate emotional expression. At this time, the emotion engine takes into account the user's latest emotional state and notifies the user, such as "Don't worry, the product is fine."

[0934] Prompt Sentence Examples

[0935] Prompt: "Check smartphones for damage before and after delivery. Choose a flexible notification method based on user sentiment."

[0936] Model input:

[0937] Previous image: "Previous image data: <image data>"

[0938] After image: "After image data: <image data>"

[0939] Emotion Data: "Emotion Data: <emotional state>"

[0940] Model output:

[0941] Damage check: "No damage."

[0942] Notification text: "Hello, customer. No damage was found before or after delivery of the product. Please rest assured."

[0943] This invention reduces users' concerns about repairs and product quality, and allows them to receive emotionally sensitive service, which is expected to improve user satisfaction.

[0944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0945] Step 1:

[0946] A user uses a smartphone to take a picture of a product or vehicle before repair. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data. The input is the image data captured by the smartphone camera and the emotion data recognized by the emotion engine, and the output is a set of these data.

[0947] Step 2:

[0948] The device sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format and the emotion data is packaged in JSON format. The input is the captured image data and emotion data, and the output is a transmission request to the cloud server.

[0949] Step 3:

[0950] The cloud server stores the received image data and emotion data in a database. When storing, it generates a unique request ID and manages it together with the image and emotion data. The input is the image data and emotion data sent from the device, and the output is the data stored in the database and the generated request ID.

[0951] Step 4:

[0952] After the repair or receiving of the product, the user uses their smartphone to take a picture of the product after repair or receiving it. At this time, the emotion engine again recognizes the user's emotion and acquires emotion data. The input is the image data captured by the smartphone camera and the recognized emotion data, and the output is a set of these data.

[0953] Step 5:

[0954] The device then sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format again, and the emotion data is packaged in JSON format. The input is the captured image data and emotion data after repair, and the output is a transmission request to the cloud server.

[0955] Step 6:

[0956] The cloud server stores the received image data and emotion data in a database and associates them with the existing request ID. The input is the repaired image data and emotion data sent from the device, and the output is the data stored in the database.

[0957] Step 7:

[0958] The cloud server sends the image data before and after the repair to the AI ​​engine to evaluate the legitimacy of the repair. The input is the image data before and after the repair stored in the database, and the output is the analysis result of the AI ​​engine.

[0959] Step 8:

[0960] The AI ​​engine compares images before and after repair to evaluate the quality and legitimacy of the repair. Specifically, it uses an image analysis algorithm to determine changes in the repair area and the accuracy of the repair. The input is image data before and after the repair, and the output is an evaluation result regarding the legitimacy of the repair.

[0961] Step 9:

[0962] The cloud server receives the evaluation results and stores them in a database. Furthermore, the emotion engine determines the user's emotional state before the notification and adjusts the notification content accordingly. The input is the AI ​​engine's evaluation results and the user's latest emotional data, and the output is the information notified to the user.

[0963] Step 10:

[0964] The cloud server sends the evaluation results to the user via email or push notification via the notification means. The notification content is appropriately adjusted based on the user's emotional state. The input is the notification information sent from the cloud server, and the output is the notification content displayed on the user's device.

[0965] This system makes it possible to provide advanced services that take into consideration the user's emotions during the repair and product quality assessment process.

[0966] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0967] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0968] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0969] [Third embodiment]

[0970] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0971] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0972] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0973] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0974] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0975] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0976] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0977] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0978] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0979] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0980] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0981] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0982] In this invention, a user first takes a photo of the exterior of their vehicle before requesting repairs and uploads the image data to a cloud server. The cloud server then stores the received image data in a database, generates a unique "request ID," and returns it to the user. This request ID is then associated with the post-repair image.

[0983] After the repair is complete, the user takes another photo of the vehicle's exterior and uploads the image data to the cloud server. The cloud server receives the post-repair image data, associates it with the request ID generated earlier, and stores it in a database.

[0984] Next, the cloud server uses AI to compare the before and after images. The AI ​​analyzes the images and evaluates the changes in the repaired area and the quality of the repair. For example, it checks whether a dent in the door before the repair is gone after the repair, and whether the paint is applied properly.

[0985] After the pass / fail result is obtained, the cloud server records the result in a database and notifies the user via email or push notification on a dedicated app.

[0986] Program processing description

[0987] 1. The user takes a photo of the car's exterior using a smartphone.

[0988] 2. The device (user's smartphone) sends an image upload request to the cloud server from a dedicated app.

[0989] 3. The server saves the received image data, generates a unique request ID, and returns it to the user.

[0990] Next, after the repair is completed, the procedure will proceed as follows:

[0991] 1. The user takes another photo of the exterior of the vehicle after repairs.

[0992] 2. The device sends a request to upload images of the repaired item to the cloud server.

[0993] 3. The server stores the received repaired images in a database and associates them with the request ID.

[0994] Next, a comparison of the before and after images begins.

[0995] 1. The server sends image data before and after repair to the AI ​​engine.

[0996] 2. AI performs image analysis to evaluate changes in the repair area and the quality of the repair.

[0997] Finally, a judgment result is generated and notified to the user.

[0998] 1. The server receives the AI ​​analysis results and stores them in a database.

[0999] 2. The server notifies the user of the result of the judgment, for example, "the repair is valid" or "the repair is incorrect."

[1000] 3. The user receives a notification via a dedicated app or email and confirms that the repair is legitimate.

[1001] Specific examples

[1002] Uploading images before requesting repair

[1003] The user takes a photo of the left side of the car using a smartphone.

[1004] The device uploads images to the cloud using a dedicated app.

[1005] The server receives the image and stores it in the database with "Request ID 12345." The Request ID is returned to the user.

[1006] Uploading images after repair is complete

[1007] The user re-photographs the left side of the repaired car.

[1008] The device uploads images of the repaired item to the cloud.

[1009] The server receives the image and stores it, linking it to request ID 12345.

[1010] Image judgment by AI

[1011] The server sends before and after images of the repair for request ID 12345 to the AI ​​engine.

[1012] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[1013] Notification of the results

[1014] The server stores the determination result that "the repair is legitimate" in a database.

[1015] The server will notify the user of the results via email or in-app push notification.

[1016] The user receives a notification and checks the details in the app.

[1017] The above is a specific embodiment of this system.

[1018] The processing flow will be explained below.

[1019] Step 1:

[1020] Before requesting repairs, the user takes a photo of the vehicle's exterior using the camera on their smartphone.

[1021] Step 2:

[1022] The device (user's smartphone) launches a dedicated app and sends a request to upload the captured images to a cloud server.

[1023] Step 3:

[1024] The server receives the uploaded image data and stores it in a database. At this time, it generates a new unique "request ID" and returns it to the user. The request ID is used to associate the before and after images of the repair.

[1025] Step 4:

[1026] After the repairs are completed, the user again uses the smartphone to take a photo of the exterior of the vehicle after repairs.

[1027] Step 5:

[1028] The device sends a request to upload the repaired image to a cloud server via a dedicated app.

[1029] Step 6:

[1030] The server receives the repaired image data and stores it in the database, linking it to the request ID generated earlier.

[1031] Step 7:

[1032] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine uses image analysis technology to evaluate the changes in the repaired area and the quality of the repair. For example, it checks whether dents have been repaired and whether the paint has been applied properly.

[1033] Step 8:

[1034] The AI ​​returns the analysis results and determines whether the repair is legitimate or illegitimate, outputting the results as a clear status such as "legitimate" or "incorrect."

[1035] Step 9:

[1036] The server stores the judgment results received from the AI ​​in a database. These judgment results are assigned a request ID.

[1037] Step 10:

[1038] The server then begins the process of notifying the user of the result of the judgment, specifically by sending an email or a push notification to a dedicated app.

[1039] Step 11:

[1040] Users receive notifications and can view detailed results within the app, allowing them to objectively determine whether the repair was legitimate.

[1041] Example 1

[1042] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1043] There is a need for a system that allows users to easily check whether their car repairs have been performed properly. Conventional methods require direct visual inspection of the condition before and after repairs, which takes time and effort. In addition, users have no way to objectively evaluate the quality of the repairs, so they may not notice if the repairs have been performed improperly. A system that solves these problems is needed.

[1044] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1045] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in storage, means for receiving post-repair images taken by the user after the repair is completed and saving the images in storage, means for analyzing the pre-repair and post-repair images using a generative AI model to determine the legitimacy of the repair, and means for notifying the user of the determination result, thereby enabling the user to quickly and accurately confirm whether the car repair was performed appropriately.

[1046] A "pre-repair image" is digital image data of the exterior of a vehicle taken by a user before repairs are made.

[1047] "Post-repair image" is digital image data of the exterior of the vehicle taken by the user after repairs have been completed.

[1048] "Storage" refers to storage devices and cloud services for storing digital data.

[1049] A "generative AI model" is an artificial intelligence model trained using machine learning techniques and used to perform a specific task, such as image analysis or judgment.

[1050] A "server" is a computer system that receives requests over a network, stores and processes data, and communicates with other computers.

[1051] "User" refers to an individual or company that uses this system to request automobile repairs.

[1052] "Analysis" is the process of using AI models to detect, compare, and evaluate the content of digital images.

[1053] The "judgment result" is information regarding the legitimacy of the repair obtained as a result of analysis by the generative AI model.

[1054] "Notification" refers to information sent from the system to the user, including emails and app push notifications.

[1055] The system of the present invention allows users to upload images taken before and after requesting a car repair to a cloud server, and then uses a generative AI model to determine the legitimacy of the repair. A specific embodiment of this system is described below.

[1056] First, the user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then selects the captured image using a dedicated app (e.g., CarRepairUploader) and presses the upload button. The device then sends this image data to a cloud server via the Internet. The cloud server receives the image data and stores it in storage (e.g., an Amazon S3 bucket). It then generates a unique "request ID" (e.g., E12345) and returns it to the user.

[1057] Next, after the repair is complete, the user takes a photo of the vehicle's exterior after repairs using the same procedure. The user then uses the dedicated app again to select the post-repair image and press the upload button. The device then sends the post-repair image data to the cloud server. The cloud server again receives the image data, associates it with the request ID (e.g., E12345) generated earlier, and stores it in storage.

[1058] The cloud server sends the image data before and after repair to a generative AI model (e.g., Amazon Rekognition). The generative AI model analyzes the images and evaluates changes in the repair area and the quality of the repair. After obtaining a judgment result, the cloud server stores the result in a database (e.g., Amazon RDS) and notifies the user. Notifications are sent via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging). Users receive a notification and can view detailed information in a dedicated app.

[1059] Specific examples

[1060] As a concrete example, the image upload before requesting repair is shown below.

[1061] Uploading images before requesting repair

[1062] The user takes a photo of the left side of the car using a smartphone (iPhone 13 Pro).

[1063] The device (iPhone 13 Pro) uploads images to the cloud server using a dedicated app (CarRepairUploader).

[1064] The server receives the image and stores it in storage (Amazon S3 bucket) with "Request ID E12345." The Request ID is returned to the user.

[1065] Uploading images after repair is complete

[1066] The user takes another photograph of the left side of the car after repairs.

[1067] The device (iPhone 13 Pro) also uploads images of the repaired item to a cloud server using a dedicated app (CarRepairUploader).

[1068] The server receives the image, associates it with the request ID E12345, and stores it in storage.

[1069] Image judgment by AI

[1070] The server sends the before and after images of the repair for request ID E12345 to the generated AI model (Amazon Rekognition).

[1071] The AI ​​engine analyzes the images and checks for changes in the repaired area, for example, to see if a dent in the door before the repair has disappeared after the repair.

[1072] Notification of the results

[1073] The server stores the determination that the repair is valid in a database (Amazon RDS).

[1074] The server notifies the user of the result, for example via email (SendGrid) or in-app push notification (Firebase Cloud Messaging).

[1075] The user receives a notification and checks the details using a dedicated app (CarRepairUploader).

[1076] Using generative AI models, the validity of vehicle repairs can be efficiently and accurately assessed. This system provides users with a means to objectively verify the quality of repairs and can prevent inappropriate repairs.

[1077] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1078] Step 1:

[1079] The user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then opens a dedicated app (e.g., CarRepairUploader), selects the image, and presses the upload button.

[1080] Input: High-resolution image data before repair taken with a smartphone

[1081] How it works: The user selects an image in the app and taps the upload button.

[1082] Output: The captured image data is saved on the device and an upload request is sent to the server.

[1083] Step 2:

[1084] The device sends the captured image data to a cloud server via the Internet.

[1085] Input: Image data before repair and request to upload to server

[1086] How it works: Your device uses its internet connection to send captured images to a cloud server.

[1087] Output: The image data arrives at the cloud server.

[1088] Step 3:

[1089] The server stores the received image data in storage (e.g., Amazon S3 bucket) and generates a unique "request ID" (e.g., E12345). The server returns the generated request ID to the user.

[1090] Input: Image data before repair submitted

[1091] Operation: Image data is stored in Amazon S3, a unique ID generation algorithm is used to generate a request ID, and the generated request ID is returned to the device.

[1092] Output: Returns the image data stored in the storage and the request ID.

[1093] Step 4:

[1094] After the repair is complete, the user takes another photo of the vehicle's exterior with their smartphone, selects the image after repair using the dedicated app, and presses the upload button.

[1095] Input: High-resolution image data of the repaired item taken with a smartphone

[1096] How it works: The user selects an image in the app and taps the upload button.

[1097] Output: The captured image data is saved on the device and an upload request is sent to the server.

[1098] Step 5:

[1099] The device then sends the repaired image data back to the cloud server via the Internet.

[1100] Input: Repaired image data and upload request to the server

[1101] How it works: Your device uses its internet connection to send captured images to a cloud server.

[1102] Output: The image data arrives at the cloud server.

[1103] Step 6:

[1104] The server stores the received repaired image data in storage (e.g., Amazon S3 bucket) and associates it with the request ID (e.g., E12345) generated earlier.

[1105] Input: The image data after repair and the existing request ID

[1106] Operation: The repaired image data is saved in Amazon S3 and linked to the existing request ID.

[1107] Output: Image data stored in storage and ID link information

[1108] Step 7:

[1109] The server sends the image data before and after the repair to a generative AI model (e.g., Amazon Rekognition), which analyzes the images and evaluates the changes in the repaired area and the quality of the repair.

[1110] Input: Before and after image data

[1111] How it works: The server sends image data to the generative AI model, which then analyzes the image.

[1112] Output: Evaluation results regarding changes in repaired areas and repair quality

[1113] Step 8:

[1114] The server receives the analysis results from the AI ​​engine, stores them in a database (e.g., Amazon RDS), and notifies the user of the results via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging).

[1115] Input: Analysis results from a generative AI model

[1116] Behavior: The analysis results are inserted into a database and the results are notified to the user using an email sending API or push notification API.

[1117] Output: Analysis results stored in a database and notifications to the user

[1118] Step 9:

[1119] Users will receive a notification and can check detailed information in a dedicated app.

[1120] Input: Notification of judgment results by email or push notification

[1121] How it works: The user opens the app and taps the "View Repair Results" button to view the results.

[1122] Output: Repair assessment results displayed in the user's app

[1123] The above is the flow of the program processing of the present invention.

[1124] (Application example 1)

[1125] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1126] Conventional technology requires users to manually take photos of the vehicle's exterior and upload the images before and after repair to a cloud server, which can lead to manual labor and errors. Furthermore, there is a lack of a means to objectively evaluate the legitimacy of repairs, making reliable damage diagnosis difficult. This makes it difficult to control the quality of repairs and achieve efficient maintenance.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1128] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, means for notifying the user of the determination result, means for automatically photographing the vehicle exterior using an on-board camera, means for uploading the image data to a cloud server, means for the cloud server to generate a unique request ID, and means for saving the photographed images in a database based on the generated request ID. This enables automatic photographing of the vehicle exterior and uploading the images to the cloud server, allowing AI to determine whether there is damage, enabling efficient and reliable damage diagnosis.

[1129] "User" refers to a person who uses the system to request repairs or inspections of a vehicle, or the owner of such a vehicle.

[1130] The "image before repair" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device before the vehicle is repaired.

[1131] A "database" is a system for accumulating, storing, and managing received image data and related information.

[1132] "Post-repair image" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device after the vehicle has been repaired.

[1133] "AI" refers to artificial intelligence technology, a general term for programs and algorithms that have the ability to analyze image data and compare before and after repairs.

[1134] A "cloud server" is a physical or virtual server used over a network that provides resources for storing and processing image data and related information.

[1135] A "request ID" is a unique identification code generated by the cloud server that is linked to a specific repair request.

[1136] "Photography device" refers to a camera or other photographic equipment that is mounted on a vehicle and automatically photographs the exterior of the vehicle.

[1137] The "judgment result" indicates the result of image analysis by AI, an evaluation of the validity of the repair and the presence or absence of damage, and its conclusion.

[1138] The "notification means" is a function for notifying the user of the judgment result, and includes email, push notification, etc.

[1139] This invention relates to a system that automatically diagnoses damage to autonomous vehicles. It eliminates the need for users to manually upload images of the vehicle before and after repairs, and also enables automatic evaluation of the quality of repairs by performing image analysis.

[1140] The system consists of the following main components:

[1141] 1. Vehicle-mounted imaging equipment:

[1142] The camera system installed in the vehicle automatically captures the exterior of the vehicle, mainly before the vehicle departs and after it returns, eliminating the need for the user to manually take photos.

[1143] 2. Cloud Server:

[1144] The captured image data is uploaded to a cloud server, which stores the received image data in a database and generates a unique request ID. The generated request ID is linked to the images before and after repair.

[1145] 3. AI image analysis engine:

[1146] AI analyzes the image data before and after repair stored on a cloud server. This AI uses image analysis technology to evaluate changes in the repaired area and the quality of the repair. Specifically, it checks whether dents and scratches that existed before the repair have been properly repaired and whether the paint has been applied accurately.

[1147] 4. Notification system:

[1148] The AI's assessment results are stored on a cloud server and notified to users and fleet managers. This notification is mainly via email or push notification, allowing users to immediately confirm the legitimacy of repairs.

[1149] Hardware and software used

[1150] Camera system: Used to capture the exterior of the vehicle. High-resolution cameras are recommended.

[1151] Cloud server: Common cloud servers such as AWS and Google Cloud are used.

[1152] Database system: A relational database such as MySQL or PostgreSQL is used.

[1153] AI image analysis software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[1154] Specific examples

[1155] A specific example of how this system can be used is shown below.

[1156] Photograph and upload before requesting repair:

[1157] The vehicle automatically takes a photo of the exterior before departing.

[1158] The vehicle system uploads the image data to a cloud server.

[1159] The cloud server receives the image data and stores it in the database as request ID 12345.

[1160] Photograph and upload after repair is complete:

[1161] After the vehicle returns from repairs, the exterior is automatically photographed again.

[1162] The vehicle system uploads post-repair image data to a cloud server.

[1163] The cloud server receives the image data and stores it, linking it to request ID 12345.

[1164] AI image assessment:

[1165] The cloud server sends image data before and after repair for request ID 12345 to the AI ​​engine.

[1166] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[1167] Notification of decision:

[1168] The cloud server stores the determination result that "the repair is legitimate" in a database.

[1169] The cloud server notifies the user of the results via email or in-app push notification.

[1170] The user receives a notification and checks the details in the app.

[1171] Prompt Sentence Examples

[1172] Develop an application that allows users to take photos of the exterior of autonomous vehicles using a smartphone and upload the images to the cloud. The application should have the ability to use AI to compare before and after images to determine the quality of the repair.

[1173] The above is a specific embodiment for carrying out the present invention.

[1174] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1175] Step 1:

[1176] A camera mounted on the vehicle automatically photographs the vehicle's exterior. The images are captured by a high-resolution camera, and metadata such as location and timestamp are also saved. (Input): Vehicle exterior. (Output): High-resolution image.

[1177] Step 2:

[1178] The device uploads the captured image data to the cloud server. At this time, the vehicle ID and other identification information are also sent. (Input): Captured image data and related identification information, (Output): Image data uploaded to the cloud server.

[1179] Step 3:

[1180] The cloud server stores the received image data in a database and generates a unique request ID, which makes it easier to identify images before and after repair. (Input): Received image data and identification information, (Output): Image data stored in the database and generated request ID.

[1181] Step 4:

[1182] After the user requests repairs, the vehicle automatically photographs its exterior once the repairs are complete. The photographic device captures the exterior of the vehicle after repairs and generates image data again. (Input): Appearance of the repaired vehicle, (Output): High-resolution image after repairs.

[1183] Step 5:

[1184] The terminal re-uploads the repaired image data to the cloud server. At this time, identification information including the vehicle ID and request ID is also sent. (Input): Repaired image data and related identification information, (Output): Upload of repaired image data to the cloud server.

[1185] Step 6:

[1186] The cloud server receives the image data after repair and stores it in a database, linking it to the request ID already saved. (Input): Received image data after repair and request ID, (Output): Image data after repair saved in the database.

[1187] Step 7:

[1188] The cloud server sends the image data before and after repair to an AI image analysis engine. This AI engine performs image analysis using generative AI models such as TensorFlow and PyTorch. (Input): Image data before and after repair. (Output): Analysis results regarding the quality of the repair.

[1189] Step 8:

[1190] The AI ​​image analysis engine compares image data before and after repair, assessing changes in each part and the quality of the repair. It determines whether the changes are appropriate and concludes that the repair is appropriate or inaccurate. (Input): Image data before and after repair, (Output): Evaluation results regarding the quality of the repair.

[1191] Step 9:

[1192] The server stores the AI ​​analysis results in a database and notifies the user. Notifications are mainly sent via email or push notifications. (Input): AI analysis results, (Output): Notification to the user.

[1193] Step 10:

[1194] The user receives a notification and can check the detailed repair quality results in the app, which allows them to know whether the repair was justified or inappropriate. (Input): Notification from the server, (Output): Display of analysis results in the user's app.

[1195] This series of processes not only allows users to easily and accurately grasp the condition of their vehicle before and after repairs, but also allows them to use the data to evaluate the quality of repairers.

[1196] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1197] The present invention provides a system that combines a process in which a user takes exterior images of a vehicle before and after requesting repairs and uploads them to a cloud server with an emotion engine that recognizes the user's emotions.

[1198] System Configuration

[1199] 1. The user uses their smartphone to take a photo of the car's exterior before repairs and uploads the image to a cloud server via a dedicated app. At this point, the emotion engine recognizes the user's emotional state and sends that data along with the image.

[1200] 2. The device (user's smartphone) generates a request to send the captured image and the user's emotional data to the cloud server and sends it.

[1201] 3. The server stores the received image data and emotion data in a database, generates a unique "request ID," and returns it to the user. The request ID is used to associate the images and emotion data before and after repair.

[1202] The procedures after the repair is completed are almost the same as those described above.

[1203] 4. After the repair is complete, the user again uses their smartphone to take a photo of the vehicle's exterior and uploads the image to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state and sends that data along with the image.

[1204] 5. The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[1205] 6. The server receives the repaired image data and emotion data, associates them with the request ID, and stores them in a database.

[1206] 7. The server uses AI to compare before and after images and evaluate the changes and quality of the repair, such as whether the dent in the door was fixed or whether the paint was applied properly.

[1207] 8. The AI ​​generates the results of the image analysis and returns them to the server, which stores them in a database.

[1208] 9. The server then initiates the procedure to notify the user of the result of the judgment. At this time, the emotion engine checks the user's emotional state before the notification and selects an appropriate notification method based on the results. For example, if the user is in an unstable emotional state, the content of the notification can be flexibly adjusted.

[1209] Specific examples

[1210] Uploading images before requesting repair

[1211] The user takes a photo of the left side of the car using their smartphone and uploads the image to a cloud server via a dedicated app. The emotion engine recognizes the user's emotions and sends emotional data such as "relief" along with the image.

[1212] The device sends a request to the cloud server. The server receives the image and emotion data and stores it in the database under "Request ID 12345."

[1213] Uploading images after repair is complete

[1214] The user then takes another photo of the left side of the repaired car and uploads it to the cloud server via a dedicated app. The emotion engine then recognizes the user's emotions again, and includes emotional data such as "expectation."

[1215] The device sends a request to the cloud server. The server receives the image and emotion data, associates it with request ID 12345, and stores it.

[1216] AI-based image judgment and emotional response

[1217] The server sends the image data before and after the repair to the AI ​​engine for analysis. The AI ​​evaluates the quality of the repair and generates a judgment that the repair is legitimate.

[1218] The server receives the judgment result and stores it in a database. The emotion engine checks the user's latest emotional state and selects an appropriate notification method (e.g., a soft-spoken notification message).

[1219] The server sends the result of the assessment to the user via email or push notification, which includes the assessment result and additional reassurance information according to the user's emotional state.

[1220] The system allows users to ensure the validity of repairs and also provides emotional consideration during the process.

[1221] The processing flow will be explained below.

[1222] Step 1:

[1223] The user uses a smartphone to take a photo of the car's exterior before repairs are made, and the image is then uploaded to a cloud server using a dedicated app.

[1224] Step 2:

[1225] The device generates and sends a request to upload the pre-repair image to a cloud server via a dedicated app. It also analyzes the user's emotional state (e.g., "relief" or "anxiety") through an emotion engine and sends that data together.

[1226] Step 3:

[1227] The server receives the uploaded image of the item before repair and the user's emotion data, stores them in a database, and generates a new "request ID" and returns it to the user.

[1228] Step 4:

[1229] After the repair is complete, the user takes another photo of the exterior of the car using their smartphone.

[1230] Step 5:

[1231] The device generates and sends a request to upload the repaired image and the user's emotion data (analyzed by the emotion engine) to the cloud server.

[1232] Step 6:

[1233] The server receives the post-repair image and emotion data and stores them in the database with the same request ID as the pre-repair image.

[1234] Step 7:

[1235] The server sends the image data before and after the repair to the AI ​​engine and requests image analysis, which then begins the process of evaluating changes in the repaired area and the quality of the repair.

[1236] Step 8:

[1237] The AI ​​engine analyzes images to determine whether repairs are warranted, for example, checking that a dent in a door has been repaired or that the paint is properly applied.

[1238] Step 9:

[1239] The AI ​​engine generates a verdict and returns the result to the server, for example, "The repair is valid" or "The repair is incorrect."

[1240] Step 10:

[1241] The server stores the received judgment results in a database. At this time, the emotion engine checks the user's latest emotional state and determines how to notify the judgment results based on that result.

[1242] Step 11:

[1243] The server then initiates a procedure to notify the user of the result of the assessment. For example, if the user is in an anxious emotional state, the server may soften the content of the notification.

[1244] Step 12:

[1245] The user receives the results via a dedicated app or email, which includes additional reassurance information tailored to the user's emotional state.

[1246] Specific examples

[1247] Uploading images before requesting repair

[1248] The user takes a photo of the left side of the car with their smartphone. The app prepares the image for upload to a cloud server. The emotion engine analyzes the user's emotion as "relief" and sends the image to the server.

[1249] The device (smartphone) sends images and emotional data to a cloud server.

[1250] Uploading repaired images

[1251] The user takes another photo of the left side of the car after the repair is complete. The app prepares to upload the repaired image to the cloud server. The emotion engine analyzes the user's emotion as "expectation" and sends it to the server.

[1252] The device (smartphone) sends images of the repaired item and emotional data to a cloud server.

[1253] AI-based image judgment and emotional response

[1254] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine performs the image analysis and generates a result that the repair is legitimate.

[1255] The server stores the result in a database and uses the emotion engine to check the user's current emotional state, which determines that the user needs to be notified in a softer tone.

[1256] The server sends the result of the assessment via email or push notification to a dedicated app, which includes additional information to reassure the user along with the result.

[1257] Users receive a notification and can check the detailed results in a dedicated app. The notification is designed with emotional consideration, giving them peace of mind regarding the repair details.

[1258] The above is a specific embodiment of the invention in which an emotion engine is combined.

[1259] Example 2

[1260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1261] While conventional car repair evaluation systems can judge the legitimacy of repairs by comparing before and after images, they lack a notification method that takes into account the emotional state of the repair requester, which has led to issues with not being able to fully achieve the requester's satisfaction or sense of security. Furthermore, conventional systems do not use emotional data for evaluation, and so the evaluation of repair companies remains one-sided.

[1262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1263] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for recognizing the user's emotional state and transmitting emotional data along with the images, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, and means for selecting an appropriate notification method and notifying the user based on the determination result and the user's latest emotional state. This not only makes it possible to objectively evaluate the legitimacy of the repair, but also increases user satisfaction and peace of mind by adopting a notification method that takes the user's emotional state into consideration.

[1264] "User" refers to any individual or corporation that uses this system.

[1265] A "terminal" is a device used by a user, such as a smartphone or tablet, that is primarily used for taking photos and transmitting data.

[1266] "Server" refers to a central processing unit that manages and stores received data and performs AI analysis and notification processing.

[1267] A "cloud server" is a server used over the Internet that includes resources for storing, processing, and analyzing data.

[1268] A "database" refers to a system that centrally manages information and allows it to be stored, searched, and updated.

[1269] An "emotion engine" refers to software or algorithms that recognize a user's emotional state and process it as data.

[1270] "Image data" refers to images of the vehicle before and after repair that are taken by the user and uploaded to the cloud server.

[1271] "Emotion data" refers to data that indicates the user's emotional state as recognized by the emotion engine.

[1272] "Request ID" refers to a unique identifier generated to identify and manage a series of data related to a repair request.

[1273] "AI engine" refers to the artificial intelligence algorithm used to analyze before and after repair images and evaluate the quality of the repair.

[1274] "Judgment result" refers to the evaluation result obtained by the AI ​​engine by comparing and analyzing images before and after repair.

[1275] "Notification means" refers to a function that sends information to the user in an appropriate manner based on the judgment result and emotion data.

[1276] A "transaction" refers to a mechanism for processing a series of operations performed within a database as a single unit and maintaining its consistency.

[1277] "HTTPS protocol" refers to an internet protocol for secure communications.

[1278] MODE FOR CARRYING OUT THE INVENTION

[1279] This invention is a system that records the process in detail when a user requests car repairs, and evaluates the quality of the repair based on images taken before and after the repair and the user's emotional state. This system is composed of the user's device, a cloud server, a database, an AI engine, and an emotion engine.

[1280] Hardware and software used

[1281] Device: A smartphone or tablet operated by the user

[1282] Cloud server: A server that stores, processes, and analyzes data (e.g., AWS, Azure)

[1283] Database: A system for storing image data and emotion data (e.g., MySQL, PostgreSQL)

[1284] Emotion engine: Software for recognizing user emotions (e.g., Emotion API, IBM Watson Tone Analyzer)

[1285] AI engine: Algorithm that analyzes before and after repair images and evaluates the quality of the repair (e.g., TensorFlow, PyTorch)

[1286] System details

[1287] Before requesting car repairs, a user takes a photo of the vehicle's condition using their smartphone and uploads the image to a cloud server using a dedicated app. At this time, the emotion engine recognizes the user's emotional state (e.g., "relief") and sends that data along with the image.

[1288] The device generates a request to send the captured image data and the user's emotion data to the cloud server, and sends it to the cloud server using the HTTPS protocol.

[1289] The server stores the received images and emotion data in a database and generates a unique request ID, which is used to associate the images and emotion data before and after repair.

[1290] After the repair is complete, the user takes another photo of the vehicle's condition with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state (e.g., "expectation") and sends that data along with the image.

[1291] The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[1292] The server receives the post-repair images and emotion data, associates them with the request ID, and stores them in a database. It then sends the pre- and post-repair image data to the AI ​​engine, requesting it to evaluate the quality of the repair.

[1293] The AI ​​engine compares image data before and after the repair, assesses changes in the repaired area, and evaluates the quality of the repair. The results are sent back to the server and stored in a database.

[1294] The emotion engine checks the user's latest emotional state before notifying them of the evaluation results and selects an appropriate notification method (e.g., soft-spoken text).

[1295] The server generates a message to notify the user based on the judgment result and emotion data, and sends it via email or push notification.

[1296] Specific examples

[1297] For example, if a user takes a photo of the left side of a car using a smartphone and uploads the image to a cloud server using a dedicated app, the emotion engine will recognize the user's emotion as "relief" and send that data along with the image.

[1298] The device sends a request to the cloud server, and the server receives the image and emotion data and stores it in the database with request ID "12345."

[1299] After the repair is complete, the user takes another photo of the left side of the vehicle with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotion as "expectation" and sends the data along with it.

[1300] The device sends a request to the cloud server, and the server receives the image and emotion data, associates it with the request ID "12345," and stores it.

[1301] The server sends the image data before and after the repair to the AI ​​engine, which evaluates the quality of the repair and generates a result that the repair is valid. The server receives the result, and the emotion engine checks the user's latest emotional state and creates a soft-spoken notification message.

[1302] The server will send an email or push notification stating that the repair is legitimate, along with information that will reassure the user.

[1303] Prompt Sentence Examples

[1304] "The user takes a photo of the vehicle before repair and uploads it to the cloud server via a dedicated app. At this time, the emotion engine recognizes the user's emotional state and sends data indicating a sense of security."

[1305] The cloud server receives the image and emotion data, generates a request ID, and returns it to the user.

[1306] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1307] Step 1:

[1308] A user uses a smartphone to take a picture of the car before repair.

[1309] Input: Use the camera function of your smartphone to take a photo of the current state of the car.

[1310] Output: Vehicle exterior image data (e.g. JPEG file).

[1311] Specific operation: The user launches the app and uses the camera function to take a photo of the area to be repaired, such as the left side of the vehicle.

[1312] Step 2:

[1313] The terminal acquires image data and the user's emotional state.

[1314] Input: Image data of the vehicle and the user's facial expression captured in real time by the camera.

[1315] Output: Vehicle image data, emotion data (e.g., relief, expectation).

[1316] Specific operation: The device's emotion engine recognizes the user's face and analyzes emotions from facial expression data. For example, it uses the Emotion API to measure feelings of relief.

[1317] Step 3:

[1318] The terminal generates and transmits a request for transmitting image data and emotion data to a cloud server.

[1319] Input: Vehicle image data, emotion data, user ID, timestamp.

[1320] Output: HTTPS outgoing request to cloud server.

[1321] Specific operation: The device generates an HTTPS request and sends it to the API endpoint of the cloud server, including image data and emotion data.

[1322] Step 4:

[1323] The server stores the received image data and emotion data in a database and generates a unique request ID.

[1324] Input: Image data, emotion data, user ID, and timestamp sent to the cloud server.

[1325] Output: The database entry and the generated request ID.

[1326] Specific operation: The server starts a transaction, records the received data in the database, and simultaneously generates a request ID and returns it to the user.

[1327] Step 5:

[1328] The server returns the request ID to the user.

[1329] Input: Generated request ID.

[1330] Output: Request ID notification to the user's device.

[1331] Specific behavior: The server generates a response containing the request ID and sends it back to the user's device via HTTPS. A notification is displayed in the user's app.

[1332] Step 6:

[1333] After the repair is completed, the user takes another picture of the repaired vehicle using their smartphone.

[1334] Input: What the vehicle will look like after repairs are complete.

[1335] Output: Image data of the vehicle's exterior after repair (e.g., JPEG file).

[1336] Specific operation: The user again uses the camera function to take a picture of the condition of the vehicle after repairs.

[1337] Step 7:

[1338] The terminal acquires the image data after repair and the user's emotional state again.

[1339] Input: A captured image of the vehicle after repair, and a user's facial expression capture.

[1340] Output: Repaired image data, emotion data (e.g., expectation).

[1341] Specific operation: The device's emotion engine performs facial recognition and analyzes and generates emotion data again.

[1342] Step 8:

[1343] The terminal generates and transmits a request for transmitting the post-repair image and emotion data to a cloud server.

[1344] Input: Repaired image data, emotion data, user ID, request ID, timestamp.

[1345] Output: HTTPS outgoing request to cloud server.

[1346] Specific operation: The device generates another HTTPS request and sends the image and emotion data to the cloud server.

[1347] Step 9:

[1348] The server receives the repaired image data and emotion data, links them to the request ID, and stores them in a database.

[1349] Input: Repaired image data, emotion data, and request ID sent to the cloud server.

[1350] Output: Updated database entry.

[1351] Specific operation: The server associates the received data based on the request ID and stores it in the database. Data integrity is ensured using transactions.

[1352] Step 10:

[1353] The server sends the image data before and after the repair to the AI ​​engine and asks it to evaluate the quality of the repair.

[1354] Input: Image data before and after repair.

[1355] Output: Data analysis request to the AI ​​engine.

[1356] Specific operation: The server accesses the AI ​​engine's API, sends image data before and after repair, and requests an evaluation.

[1357] Step 11:

[1358] The AI ​​engine analyzes the image data and evaluates changes in the repair area and the quality of the repair.

[1359] Input: Image data before and after repair.

[1360] Output: Evaluation result data (e.g., "Repair is justified").

[1361] How it works: The AI ​​engine applies image analysis algorithms to evaluate changes between images to determine the quality of the repair, and generates an assessment result that is sent back to the server.

[1362] Step 12:

[1363] The server receives the evaluation results from the AI ​​engine and stores them in a database.

[1364] Input: Evaluation result data sent from the AI ​​engine.

[1365] Output: Evaluation results stored in a database.

[1366] Specific operation: The received evaluation results are associated with the request ID and stored in the database.

[1367] Step 13:

[1368] The emotion engine checks the user's latest emotional state and selects the appropriate notification method.

[1369] Input: The user's latest emotion data.

[1370] Output: Select an appropriate notification method (e.g., a soft-spoken notification).

[1371] Specific operation: The emotion engine analyzes the user's latest emotion data and determines the appropriate notification method and content.

[1372] Step 14:

[1373] The server generates a notification message based on the determination result and the selected notification method and sends it to the user.

[1374] Input: Evaluation result data, emotion data, selected notification method.

[1375] Output: A notification message to the user.

[1376] Specific operation: The server converts the judgment result into a soft-spoken sentence and sends it to the user as an email or push notification.

[1377] (Application example 2)

[1378] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1379] Conventional systems can check the vehicle's condition before and after repair using images and use AI to determine the legitimacy of the repair, but they do not take into account the user's emotional state. As a result, users often feel anxious or dissatisfied with the repair process. In particular, similar problems have occurred when determining the quality of products purchased on online shopping sites, causing a decrease in user satisfaction. The objective of this invention is to solve these problems and provide a system that can provide notifications and evaluations that take into account the user's emotions.

[1380] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a pre-repair image taken by the user and saving the image in a database, means for receiving a post-repair image taken by the user after the repair is completed and saving the image in a database, means for comparing the pre-repair and post-repair images using AI and determining the legitimacy of the repair, means for recognizing the user's emotional state and sending the emotional data along with the image to a cloud server, means for checking the user's emotional state before notification and adjusting the notification content based on that, and means for notifying the user of the determination result. This makes it possible to provide a system that takes into consideration not only repair and product quality assessment, but also the user's emotions.

[1381] "User" means an individual or corporation that uses the system to repair and assess the quality of vehicles or products.

[1382] "Image before repair" is image data showing the exterior of a vehicle or product taken by a user before requesting repair.

[1383] "Post-repair image" is image data showing the exterior of a vehicle or product taken after the user has requested repairs.

[1384] The "database" is an information management system that stores information processed and saved by the system, such as image data before and after repair and emotional data.

[1385] A "cloud server" is a remote server that processes, stores, and manages data over the Internet.

[1386] "AI" stands for artificial intelligence, a technology that uses machine learning and deep learning to analyze image data and determine the legitimacy of repairs.

[1387] An "emotion engine" is a software or hardware system for recognizing a user's emotional state and capturing that data.

[1388] "Emotion data" is data that indicates the emotional state of the user as recognized by the emotion engine.

[1389] The "judgment result" is data showing the results of the AI's analysis of pre-repair and post-repair images and its evaluation of the legitimacy of the repair.

[1390] "Notification means" refers to the method, such as email or push notification, that the system uses to notify the user of the judgment results.

[1391] In this invention, a system is implemented in which a user takes a photo of the exterior of a vehicle or product they have purchased or requested repairs on their smartphone, and then sends the image and emotional data to a cloud server. This system uses AI to analyze the images before and after the repair and evaluate the legitimacy of the repair. It also adjusts the notification content based on the user's emotional state and appropriately communicates the results to the user.

[1392] System Configuration

[1393] User terminal

[1394] Users use their smartphones to take pictures of their vehicles or products before and after repairs. The emotion engine recognizes the user's emotional state when taking the pictures and captures that data. This emotional data is then sent to a cloud server along with the images.

[1395] Cloud Server

[1396] The cloud server stores the received image data and emotional data in a database. Furthermore, the AI ​​engine compares the images before and after the repair to evaluate the legitimacy of the repair. Based on the evaluation results and the user's emotional data, the results are communicated to the user via a notification method. In particular, the system evaluates the user's emotional state before the notification and adjusts the notification content accordingly.

[1397] Hardware and software used

[1398] Smartphone: A device that allows users to take pictures, acquire emotion data, and send it to a cloud server.

[1399] OpenCV: A library for capturing images using a smartphone's camera function.

[1400] Emotion Engine: Uses emotion recognition software to analyze the user's emotional state in real time.

[1401] Cloud Server: A remote server that receives and stores data and analyzes it using AI models.

[1402] AI Engine: Artificial intelligence technology that analyzes received images and determines repair and product quality.

[1403] Notification service: A service for notifying users of results via email or push notification.

[1404] Specific examples

[1405] For example, a user orders a new smartphone from an online store and takes a picture of its packaging before receiving it. When the picture is uploaded to a cloud server via a dedicated app, the emotion engine recognizes the user's emotion as "anticipation" and sends the emotion data along with it. Later, when the user takes another picture after receiving the product and uploads it to the cloud server in the same way, the emotion engine recognizes the user's emotion as "excitement."

[1406] The cloud server sends before and after repair images to the AI ​​engine, which judges the quality of the product. As a result, it judges that "the product was delivered without any problems" and notifies the user of the judgment result using an appropriate emotional expression. At this time, the emotion engine takes into account the user's latest emotional state and notifies the user, such as "Don't worry, the product is fine."

[1407] Prompt Sentence Examples

[1408] Prompt: "Check smartphones for damage before and after delivery. Choose a flexible notification method based on user sentiment."

[1409] Model input:

[1410] Previous image: "Previous image data: <image data>"

[1411] After image: "After image data: <image data>"

[1412] Emotion Data: "Emotion Data: <emotional state>"

[1413] Model output:

[1414] Damage check: "No damage."

[1415] Notification text: "Hello, customer. No damage was found before or after delivery of the product. Please rest assured."

[1416] This invention reduces users' concerns about repairs and product quality, and allows them to receive emotionally sensitive service, which is expected to improve user satisfaction.

[1417] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1418] Step 1:

[1419] A user uses a smartphone to take a picture of a product or vehicle before repair. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data. The input is the image data captured by the smartphone camera and the emotion data recognized by the emotion engine, and the output is a set of these data.

[1420] Step 2:

[1421] The device sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format and the emotion data is packaged in JSON format. The input is the captured image data and emotion data, and the output is a transmission request to the cloud server.

[1422] Step 3:

[1423] The cloud server stores the received image data and emotion data in a database. When storing, it generates a unique request ID and manages it together with the image and emotion data. The input is the image data and emotion data sent from the device, and the output is the data stored in the database and the generated request ID.

[1424] Step 4:

[1425] After the repair or receiving of the product, the user uses their smartphone to take a picture of the product after repair or receiving it. At this time, the emotion engine again recognizes the user's emotion and acquires emotion data. The input is the image data captured by the smartphone camera and the recognized emotion data, and the output is a set of these data.

[1426] Step 5:

[1427] The device then sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format again, and the emotion data is packaged in JSON format. The input is the captured image data and emotion data after repair, and the output is a transmission request to the cloud server.

[1428] Step 6:

[1429] The cloud server stores the received image data and emotion data in a database and associates them with the existing request ID. The input is the repaired image data and emotion data sent from the device, and the output is the data stored in the database.

[1430] Step 7:

[1431] The cloud server sends the image data before and after the repair to the AI ​​engine to evaluate the legitimacy of the repair. The input is the image data before and after the repair stored in the database, and the output is the analysis result of the AI ​​engine.

[1432] Step 8:

[1433] The AI ​​engine compares images before and after repair to evaluate the quality and legitimacy of the repair. Specifically, it uses an image analysis algorithm to determine changes in the repair area and the accuracy of the repair. The input is image data before and after the repair, and the output is an evaluation result regarding the legitimacy of the repair.

[1434] Step 9:

[1435] The cloud server receives the evaluation results and stores them in a database. Furthermore, the emotion engine determines the user's emotional state before the notification and adjusts the notification content accordingly. The input is the AI ​​engine's evaluation results and the user's latest emotional data, and the output is the information notified to the user.

[1436] Step 10:

[1437] The cloud server sends the evaluation results to the user via email or push notification via the notification means. The notification content is appropriately adjusted based on the user's emotional state. The input is the notification information sent from the cloud server, and the output is the notification content displayed on the user's device.

[1438] This system makes it possible to provide advanced services that take into consideration the user's emotions during the repair and product quality assessment process.

[1439] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1440] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1441] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1442] [Fourth embodiment]

[1443] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1444] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1445] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1446] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1447] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1449] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1450] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1451] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1452] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1453] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1454] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1455] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1456] In this invention, a user first takes a photo of the exterior of their vehicle before requesting repairs and uploads the image data to a cloud server. The cloud server then stores the received image data in a database, generates a unique "request ID," and returns it to the user. This request ID is then associated with the post-repair image.

[1457] After the repair is complete, the user takes another photo of the vehicle's exterior and uploads the image data to the cloud server. The cloud server receives the post-repair image data, associates it with the request ID generated earlier, and stores it in a database.

[1458] Next, the cloud server uses AI to compare the before and after images. The AI ​​analyzes the images and evaluates the changes in the repaired area and the quality of the repair. For example, it checks whether a dent in the door before the repair is gone after the repair, and whether the paint is applied properly.

[1459] After the pass / fail result is obtained, the cloud server records the result in a database and notifies the user via email or push notification on a dedicated app.

[1460] Program processing description

[1461] 1. The user takes a photo of the car's exterior using a smartphone.

[1462] 2. The device (user's smartphone) sends an image upload request to the cloud server from a dedicated app.

[1463] 3. The server saves the received image data, generates a unique request ID, and returns it to the user.

[1464] Next, after the repair is completed, the procedure will proceed as follows:

[1465] 1. The user takes another photo of the exterior of the vehicle after repairs.

[1466] 2. The device sends a request to upload images of the repaired item to the cloud server.

[1467] 3. The server stores the received repaired images in a database and associates them with the request ID.

[1468] Next, a comparison of the before and after images begins.

[1469] 1. The server sends image data before and after repair to the AI ​​engine.

[1470] 2. AI performs image analysis to evaluate changes in the repair area and the quality of the repair.

[1471] Finally, a judgment result is generated and notified to the user.

[1472] 1. The server receives the AI ​​analysis results and stores them in a database.

[1473] 2. The server notifies the user of the result of the judgment, for example, "the repair is valid" or "the repair is incorrect."

[1474] 3. The user receives a notification via a dedicated app or email and confirms that the repair is legitimate.

[1475] Specific examples

[1476] Uploading images before requesting repair

[1477] The user takes a photo of the left side of the car using a smartphone.

[1478] The device uploads images to the cloud using a dedicated app.

[1479] The server receives the image and stores it in the database with "Request ID 12345." The Request ID is returned to the user.

[1480] Uploading images after repair is complete

[1481] The user re-photographs the left side of the repaired car.

[1482] The device uploads images of the repaired item to the cloud.

[1483] The server receives the image and stores it, linking it to request ID 12345.

[1484] Image judgment by AI

[1485] The server sends before and after images of the repair for request ID 12345 to the AI ​​engine.

[1486] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[1487] Notification of the results

[1488] The server stores the determination result that "the repair is legitimate" in a database.

[1489] The server will notify the user of the results via email or in-app push notification.

[1490] The user receives a notification and checks the details in the app.

[1491] The above is a specific embodiment of this system.

[1492] The processing flow will be explained below.

[1493] Step 1:

[1494] Before requesting repairs, the user takes a photo of the vehicle's exterior using the camera on their smartphone.

[1495] Step 2:

[1496] The device (user's smartphone) launches a dedicated app and sends a request to upload the captured images to a cloud server.

[1497] Step 3:

[1498] The server receives the uploaded image data and stores it in a database. At this time, it generates a new unique "request ID" and returns it to the user. The request ID is used to associate the before and after images of the repair.

[1499] Step 4:

[1500] After the repairs are completed, the user again uses the smartphone to take a photo of the exterior of the vehicle after repairs.

[1501] Step 5:

[1502] The device sends a request to upload the repaired image to a cloud server via a dedicated app.

[1503] Step 6:

[1504] The server receives the repaired image data and stores it in the database, linking it to the request ID generated earlier.

[1505] Step 7:

[1506] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine uses image analysis technology to evaluate the changes in the repaired area and the quality of the repair. For example, it checks whether dents have been repaired and whether the paint has been applied properly.

[1507] Step 8:

[1508] The AI ​​returns the analysis results and determines whether the repair is legitimate or illegitimate, outputting the results as a clear status such as "legitimate" or "incorrect."

[1509] Step 9:

[1510] The server stores the judgment results received from the AI ​​in a database. These judgment results are assigned a request ID.

[1511] Step 10:

[1512] The server then begins the process of notifying the user of the result of the judgment, specifically by sending an email or a push notification to a dedicated app.

[1513] Step 11:

[1514] Users receive notifications and can view detailed results within the app, allowing them to objectively determine whether the repair was legitimate.

[1515] Example 1

[1516] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1517] There is a need for a system that allows users to easily check whether their car repairs have been performed properly. Conventional methods require direct visual inspection of the condition before and after repairs, which takes time and effort. In addition, users have no way to objectively evaluate the quality of the repairs, so they may not notice if the repairs have been performed improperly. A system that solves these problems is needed.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1519] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in storage, means for receiving post-repair images taken by the user after the repair is completed and saving the images in storage, means for analyzing the pre-repair and post-repair images using a generative AI model to determine the legitimacy of the repair, and means for notifying the user of the determination result, thereby enabling the user to quickly and accurately confirm whether the car repair was performed appropriately.

[1520] A "pre-repair image" is digital image data of the exterior of a vehicle taken by a user before repairs are made.

[1521] "Post-repair image" is digital image data of the exterior of the vehicle taken by the user after repairs have been completed.

[1522] "Storage" refers to storage devices and cloud services for storing digital data.

[1523] A "generative AI model" is an artificial intelligence model trained using machine learning techniques and used to perform a specific task, such as image analysis or judgment.

[1524] A "server" is a computer system that receives requests over a network, stores and processes data, and communicates with other computers.

[1525] "User" refers to an individual or company that uses this system to request automobile repairs.

[1526] "Analysis" is the process of using AI models to detect, compare, and evaluate the content of digital images.

[1527] The "judgment result" is information regarding the legitimacy of the repair obtained as a result of analysis by the generative AI model.

[1528] "Notification" refers to information sent from the system to the user, including emails and app push notifications.

[1529] The system of the present invention allows users to upload images taken before and after requesting a car repair to a cloud server, and then uses a generative AI model to determine the legitimacy of the repair. A specific embodiment of this system is described below.

[1530] First, the user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then selects the captured image using a dedicated app (e.g., CarRepairUploader) and presses the upload button. The device then sends this image data to a cloud server via the Internet. The cloud server receives the image data and stores it in storage (e.g., an Amazon S3 bucket). It then generates a unique "request ID" (e.g., E12345) and returns it to the user.

[1531] Next, after the repair is complete, the user takes a photo of the vehicle's exterior after repairs using the same procedure. The user then uses the dedicated app again to select the post-repair image and press the upload button. The device then sends the post-repair image data to the cloud server. The cloud server again receives the image data, associates it with the request ID (e.g., E12345) generated earlier, and stores it in storage.

[1532] The cloud server sends the image data before and after repair to a generative AI model (e.g., Amazon Rekognition). The generative AI model analyzes the images and evaluates changes in the repair area and the quality of the repair. After obtaining a judgment result, the cloud server stores the result in a database (e.g., Amazon RDS) and notifies the user. Notifications are sent via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging). Users receive a notification and can view detailed information in a dedicated app.

[1533] Specific examples

[1534] As a concrete example, the image upload before requesting repair is shown below.

[1535] Uploading images before requesting repair

[1536] The user takes a photo of the left side of the car using a smartphone (iPhone 13 Pro).

[1537] The device (iPhone 13 Pro) uploads images to the cloud server using a dedicated app (CarRepairUploader).

[1538] The server receives the image and stores it in storage (Amazon S3 bucket) with "Request ID E12345." The Request ID is returned to the user.

[1539] Uploading images after repair is complete

[1540] The user takes another photograph of the left side of the car after repairs.

[1541] The device (iPhone 13 Pro) also uploads images of the repaired item to a cloud server using a dedicated app (CarRepairUploader).

[1542] The server receives the image, associates it with the request ID E12345, and stores it in storage.

[1543] Image judgment by AI

[1544] The server sends the before and after images of the repair for request ID E12345 to the generated AI model (Amazon Rekognition).

[1545] The AI ​​engine analyzes the images and checks for changes in the repaired area, for example, to see if a dent in the door before the repair has disappeared after the repair.

[1546] Notification of the results

[1547] The server stores the determination that the repair is valid in a database (Amazon RDS).

[1548] The server notifies the user of the result, for example via email (SendGrid) or in-app push notification (Firebase Cloud Messaging).

[1549] The user receives a notification and checks the details using a dedicated app (CarRepairUploader).

[1550] Using generative AI models, the validity of vehicle repairs can be efficiently and accurately assessed. This system provides users with a means to objectively verify the quality of repairs and can prevent inappropriate repairs.

[1551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1552] Step 1:

[1553] The user uses a smartphone (e.g., iPhone 13 Pro) to take a photo of the vehicle's exterior before repair. The user then opens a dedicated app (e.g., CarRepairUploader), selects the image, and presses the upload button.

[1554] Input: High-resolution image data before repair taken with a smartphone

[1555] How it works: The user selects an image in the app and taps the upload button.

[1556] Output: The captured image data is saved on the device and an upload request is sent to the server.

[1557] Step 2:

[1558] The device sends the captured image data to a cloud server via the Internet.

[1559] Input: Image data before repair and request to upload to server

[1560] How it works: Your device uses its internet connection to send captured images to a cloud server.

[1561] Output: The image data arrives at the cloud server.

[1562] Step 3:

[1563] The server stores the received image data in storage (e.g., Amazon S3 bucket) and generates a unique "request ID" (e.g., E12345). The server returns the generated request ID to the user.

[1564] Input: Image data before repair submitted

[1565] Operation: Image data is stored in Amazon S3, a unique ID generation algorithm is used to generate a request ID, and the generated request ID is returned to the device.

[1566] Output: Returns the image data stored in the storage and the request ID.

[1567] Step 4:

[1568] After the repair is complete, the user takes another photo of the vehicle's exterior with their smartphone, selects the image after repair using the dedicated app, and presses the upload button.

[1569] Input: High-resolution image data of the repaired item taken with a smartphone

[1570] How it works: The user selects an image in the app and taps the upload button.

[1571] Output: The captured image data is saved on the device and an upload request is sent to the server.

[1572] Step 5:

[1573] The device then sends the repaired image data back to the cloud server via the Internet.

[1574] Input: Repaired image data and upload request to the server

[1575] How it works: Your device uses its internet connection to send captured images to a cloud server.

[1576] Output: The image data arrives at the cloud server.

[1577] Step 6:

[1578] The server stores the received repaired image data in storage (e.g., Amazon S3 bucket) and associates it with the request ID (e.g., E12345) generated earlier.

[1579] Input: The image data after repair and the existing request ID

[1580] Operation: The repaired image data is saved in Amazon S3 and linked to the existing request ID.

[1581] Output: Image data stored in storage and ID link information

[1582] Step 7:

[1583] The server sends the image data before and after the repair to a generative AI model (e.g., Amazon Rekognition), which analyzes the images and evaluates the changes in the repaired area and the quality of the repair.

[1584] Input: Before and after image data

[1585] How it works: The server sends image data to the generative AI model, which then analyzes the image.

[1586] Output: Evaluation results regarding changes in repaired areas and repair quality

[1587] Step 8:

[1588] The server receives the analysis results from the AI ​​engine, stores them in a database (e.g., Amazon RDS), and notifies the user of the results via email (e.g., SendGrid) or in-app push notifications (e.g., Firebase Cloud Messaging).

[1589] Input: Analysis results from a generative AI model

[1590] Behavior: The analysis results are inserted into a database and the results are notified to the user using an email sending API or push notification API.

[1591] Output: Analysis results stored in a database and notifications to the user

[1592] Step 9:

[1593] Users will receive a notification and can check detailed information in a dedicated app.

[1594] Input: Notification of judgment results by email or push notification

[1595] How it works: The user opens the app and taps the "View Repair Results" button to view the results.

[1596] Output: Repair assessment results displayed in the user's app

[1597] The above is the flow of the program processing of the present invention.

[1598] (Application example 1)

[1599] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1600] Conventional technology requires users to manually take photos of the vehicle's exterior and upload the images before and after repair to a cloud server, which can lead to manual labor and errors. Furthermore, there is a lack of a means to objectively evaluate the legitimacy of repairs, making reliable damage diagnosis difficult. This makes it difficult to control the quality of repairs and achieve efficient maintenance.

[1601] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1602] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, means for notifying the user of the determination result, means for automatically photographing the vehicle exterior using an on-board camera, means for uploading the image data to a cloud server, means for the cloud server to generate a unique request ID, and means for saving the photographed images in a database based on the generated request ID. This enables automatic photographing of the vehicle exterior and uploading the images to the cloud server, allowing AI to determine whether there is damage, enabling efficient and reliable damage diagnosis.

[1603] "User" refers to a person who uses the system to request repairs or inspections of a vehicle, or the owner of such a vehicle.

[1604] The "image before repair" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device before the vehicle is repaired.

[1605] A "database" is a system for accumulating, storing, and managing received image data and related information.

[1606] "Post-repair image" is photographic data of the exterior of the vehicle taken by a user or an automatic photographing device after the vehicle has been repaired.

[1607] "AI" refers to artificial intelligence technology, a general term for programs and algorithms that have the ability to analyze image data and compare before and after repairs.

[1608] A "cloud server" is a physical or virtual server used over a network that provides resources for storing and processing image data and related information.

[1609] A "request ID" is a unique identification code generated by the cloud server that is linked to a specific repair request.

[1610] "Photography device" refers to a camera or other photographic equipment that is mounted on a vehicle and automatically photographs the exterior of the vehicle.

[1611] The "judgment result" indicates the result of image analysis by AI, an evaluation of the validity of the repair and the presence or absence of damage, and its conclusion.

[1612] The "notification means" is a function for notifying the user of the judgment result, and includes email, push notification, etc.

[1613] This invention relates to a system that automatically diagnoses damage to autonomous vehicles. It eliminates the need for users to manually upload images of the vehicle before and after repairs, and also enables automatic evaluation of the quality of repairs by performing image analysis.

[1614] The system consists of the following main components:

[1615] 1. Vehicle-mounted imaging equipment:

[1616] The camera system installed in the vehicle automatically captures the exterior of the vehicle, mainly before the vehicle departs and after it returns, eliminating the need for the user to manually take photos.

[1617] 2. Cloud Server:

[1618] The captured image data is uploaded to a cloud server, which stores the received image data in a database and generates a unique request ID. The generated request ID is linked to the images before and after repair.

[1619] 3. AI image analysis engine:

[1620] AI analyzes the image data before and after repair stored on a cloud server. This AI uses image analysis technology to evaluate changes in the repaired area and the quality of the repair. Specifically, it checks whether dents and scratches that existed before the repair have been properly repaired and whether the paint has been applied accurately.

[1621] 4. Notification system:

[1622] The AI's assessment results are stored on a cloud server and notified to users and fleet managers. This notification is mainly via email or push notification, allowing users to immediately confirm the legitimacy of repairs.

[1623] Hardware and software used

[1624] Camera system: Used to capture the exterior of the vehicle. High-resolution cameras are recommended.

[1625] Cloud server: Common cloud servers such as AWS and Google Cloud are used.

[1626] Database system: A relational database such as MySQL or PostgreSQL is used.

[1627] AI image analysis software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[1628] Specific examples

[1629] A specific example of how this system can be used is shown below.

[1630] Photograph and upload before requesting repair:

[1631] The vehicle automatically takes a photo of the exterior before departing.

[1632] The vehicle system uploads the image data to a cloud server.

[1633] The cloud server receives the image data and stores it in the database as request ID 12345.

[1634] Photograph and upload after repair is complete:

[1635] After the vehicle returns from repairs, the exterior is automatically photographed again.

[1636] The vehicle system uploads post-repair image data to a cloud server.

[1637] The cloud server receives the image data and stores it, linking it to request ID 12345.

[1638] AI image assessment:

[1639] The cloud server sends image data before and after repair for request ID 12345 to the AI ​​engine.

[1640] AI analyzes images to check for changes in the repaired area, for example, to see if a dent in a door before the repair has disappeared after the repair.

[1641] Notification of decision:

[1642] The cloud server stores the determination result that "the repair is legitimate" in a database.

[1643] The cloud server notifies the user of the results via email or in-app push notification.

[1644] The user receives a notification and checks the details in the app.

[1645] Prompt Sentence Examples

[1646] Develop an application that allows users to take photos of the exterior of autonomous vehicles using a smartphone and upload the images to the cloud. The application should have the ability to use AI to compare before and after images to determine the quality of the repair.

[1647] The above is a specific embodiment for carrying out the present invention.

[1648] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1649] Step 1:

[1650] A camera mounted on the vehicle automatically photographs the vehicle's exterior. The images are captured by a high-resolution camera, and metadata such as location and timestamp are also saved. (Input): Vehicle exterior. (Output): High-resolution image.

[1651] Step 2:

[1652] The device uploads the captured image data to the cloud server. At this time, the vehicle ID and other identification information are also sent. (Input): Captured image data and related identification information, (Output): Image data uploaded to the cloud server.

[1653] Step 3:

[1654] The cloud server stores the received image data in a database and generates a unique request ID, which makes it easier to identify images before and after repair. (Input): Received image data and identification information, (Output): Image data stored in the database and generated request ID.

[1655] Step 4:

[1656] After the user requests repairs, the vehicle automatically photographs its exterior once the repairs are complete. The photographic device captures the exterior of the vehicle after repairs and generates image data again. (Input): Appearance of the repaired vehicle, (Output): High-resolution image after repairs.

[1657] Step 5:

[1658] The terminal re-uploads the repaired image data to the cloud server. At this time, identification information including the vehicle ID and request ID is also sent. (Input): Repaired image data and related identification information, (Output): Upload of repaired image data to the cloud server.

[1659] Step 6:

[1660] The cloud server receives the image data after repair and stores it in a database, linking it to the request ID already saved. (Input): Received image data after repair and request ID, (Output): Image data after repair saved in the database.

[1661] Step 7:

[1662] The cloud server sends the image data before and after repair to an AI image analysis engine. This AI engine performs image analysis using generative AI models such as TensorFlow and PyTorch. (Input): Image data before and after repair. (Output): Analysis results regarding the quality of the repair.

[1663] Step 8:

[1664] The AI ​​image analysis engine compares image data before and after repair, assessing changes in each part and the quality of the repair. It determines whether the changes are appropriate and concludes that the repair is appropriate or inaccurate. (Input): Image data before and after repair, (Output): Evaluation results regarding the quality of the repair.

[1665] Step 9:

[1666] The server stores the AI ​​analysis results in a database and notifies the user. Notifications are mainly sent via email or push notifications. (Input): AI analysis results, (Output): Notification to the user.

[1667] Step 10:

[1668] The user receives a notification and can check the detailed repair quality results in the app, which allows them to know whether the repair was justified or inappropriate. (Input): Notification from the server, (Output): Display of analysis results in the user's app.

[1669] This series of processes not only allows users to easily and accurately grasp the condition of their vehicle before and after repairs, but also allows them to use the data to evaluate the quality of repairers.

[1670] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1671] The present invention provides a system that combines a process in which a user takes exterior images of a vehicle before and after requesting repairs and uploads them to a cloud server with an emotion engine that recognizes the user's emotions.

[1672] System Configuration

[1673] 1. The user uses their smartphone to take a photo of the car's exterior before repairs and uploads the image to a cloud server via a dedicated app. At this point, the emotion engine recognizes the user's emotional state and sends that data along with the image.

[1674] 2. The device (user's smartphone) generates a request to send the captured image and the user's emotional data to the cloud server and sends it.

[1675] 3. The server stores the received image data and emotion data in a database, generates a unique "request ID," and returns it to the user. The request ID is used to associate the images and emotion data before and after repair.

[1676] The procedures after the repair is completed are almost the same as those described above.

[1677] 4. After the repair is complete, the user again uses their smartphone to take a photo of the vehicle's exterior and uploads the image to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state and sends that data along with the image.

[1678] 5. The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[1679] 6. The server receives the repaired image data and emotion data, associates them with the request ID, and stores them in a database.

[1680] 7. The server uses AI to compare before and after images and evaluate the changes and quality of the repair, such as whether the dent in the door was fixed or whether the paint was applied properly.

[1681] 8. The AI ​​generates the results of the image analysis and returns them to the server, which stores them in a database.

[1682] 9. The server then initiates the procedure to notify the user of the result of the judgment. At this time, the emotion engine checks the user's emotional state before the notification and selects an appropriate notification method based on the results. For example, if the user is in an unstable emotional state, the content of the notification can be flexibly adjusted.

[1683] Specific examples

[1684] Uploading images before requesting repair

[1685] The user takes a photo of the left side of the car using their smartphone and uploads the image to a cloud server via a dedicated app. The emotion engine recognizes the user's emotions and sends emotional data such as "relief" along with the image.

[1686] The device sends a request to the cloud server. The server receives the image and emotion data and stores it in the database under "Request ID 12345."

[1687] Uploading images after repair is complete

[1688] The user then takes another photo of the left side of the repaired car and uploads it to the cloud server via a dedicated app. The emotion engine then recognizes the user's emotions again, and includes emotional data such as "expectation."

[1689] The device sends a request to the cloud server. The server receives the image and emotion data, associates it with request ID 12345, and stores it.

[1690] AI-based image judgment and emotional response

[1691] The server sends the image data before and after the repair to the AI ​​engine for analysis. The AI ​​evaluates the quality of the repair and generates a judgment that the repair is legitimate.

[1692] The server receives the judgment result and stores it in a database. The emotion engine checks the user's latest emotional state and selects an appropriate notification method (e.g., a soft-spoken notification message).

[1693] The server sends the result of the assessment to the user via email or push notification, which includes the assessment result and additional reassurance information according to the user's emotional state.

[1694] The system allows users to ensure the validity of repairs and also provides emotional consideration during the process.

[1695] The processing flow will be explained below.

[1696] Step 1:

[1697] The user uses a smartphone to take a photo of the car's exterior before repairs are made, and the image is then uploaded to a cloud server using a dedicated app.

[1698] Step 2:

[1699] The device generates and sends a request to upload the pre-repair image to a cloud server via a dedicated app. It also analyzes the user's emotional state (e.g., "relief" or "anxiety") through an emotion engine and sends that data together.

[1700] Step 3:

[1701] The server receives the uploaded image of the item before repair and the user's emotion data, stores them in a database, and generates a new "request ID" and returns it to the user.

[1702] Step 4:

[1703] After the repair is complete, the user takes another photo of the exterior of the car using their smartphone.

[1704] Step 5:

[1705] The device generates and sends a request to upload the repaired image and the user's emotion data (analyzed by the emotion engine) to the cloud server.

[1706] Step 6:

[1707] The server receives the post-repair image and emotion data and stores them in the database with the same request ID as the pre-repair image.

[1708] Step 7:

[1709] The server sends the image data before and after the repair to the AI ​​engine and requests image analysis, which then begins the process of evaluating changes in the repaired area and the quality of the repair.

[1710] Step 8:

[1711] The AI ​​engine analyzes images to determine whether repairs are warranted, for example, checking that a dent in a door has been repaired or that the paint is properly applied.

[1712] Step 9:

[1713] The AI ​​engine generates a verdict and returns the result to the server, for example, "The repair is valid" or "The repair is incorrect."

[1714] Step 10:

[1715] The server stores the received judgment results in a database. At this time, the emotion engine checks the user's latest emotional state and determines how to notify the judgment results based on that result.

[1716] Step 11:

[1717] The server then initiates a procedure to notify the user of the result of the assessment. For example, if the user is in an anxious emotional state, the server may soften the content of the notification.

[1718] Step 12:

[1719] The user receives the results via a dedicated app or email, which includes additional reassurance information tailored to the user's emotional state.

[1720] Specific examples

[1721] Uploading images before requesting repair

[1722] The user takes a photo of the left side of the car with their smartphone. The app prepares the image for upload to a cloud server. The emotion engine analyzes the user's emotion as "relief" and sends the image to the server.

[1723] The device (smartphone) sends images and emotional data to a cloud server.

[1724] Uploading repaired images

[1725] The user takes another photo of the left side of the car after the repair is complete. The app prepares to upload the repaired image to the cloud server. The emotion engine analyzes the user's emotion as "expectation" and sends it to the server.

[1726] The device (smartphone) sends images of the repaired item and emotional data to a cloud server.

[1727] AI-based image judgment and emotional response

[1728] The server sends the image data before and after the repair to the AI ​​engine and requests an analysis. The AI ​​engine performs the image analysis and generates a result that the repair is legitimate.

[1729] The server stores the result in a database and uses the emotion engine to check the user's current emotional state, which determines that the user needs to be notified in a softer tone.

[1730] The server sends the result of the assessment via email or push notification to a dedicated app, which includes additional information to reassure the user along with the result.

[1731] Users receive a notification and can check the detailed results in a dedicated app. The notification is designed with emotional consideration, giving them peace of mind regarding the repair details.

[1732] The above is a specific embodiment of the invention in which an emotion engine is combined.

[1733] Example 2

[1734] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1735] While conventional car repair evaluation systems can judge the legitimacy of repairs by comparing before and after images, they lack a notification method that takes into account the emotional state of the repair requester, which has led to issues with not being able to fully achieve the requester's satisfaction or sense of security. Furthermore, conventional systems do not use emotional data for evaluation, and so the evaluation of repair companies remains one-sided.

[1736] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1737] In this invention, the server includes means for receiving pre-repair images taken by the user and saving the images in a database, means for recognizing the user's emotional state and transmitting emotional data along with the images, means for receiving post-repair images taken by the user after the repair is completed and saving the images in a database, means for comparing the pre-repair and post-repair images using AI to determine the legitimacy of the repair, and means for selecting an appropriate notification method and notifying the user based on the determination result and the user's latest emotional state. This not only makes it possible to objectively evaluate the legitimacy of the repair, but also increases user satisfaction and peace of mind by adopting a notification method that takes the user's emotional state into consideration.

[1738] "User" refers to any individual or corporation that uses this system.

[1739] A "terminal" is a device used by a user, such as a smartphone or tablet, that is primarily used for taking photos and transmitting data.

[1740] "Server" refers to a central processing unit that manages and stores received data and performs AI analysis and notification processing.

[1741] A "cloud server" is a server used over the Internet that includes resources for storing, processing, and analyzing data.

[1742] A "database" refers to a system that centrally manages information and allows it to be stored, searched, and updated.

[1743] An "emotion engine" refers to software or algorithms that recognize a user's emotional state and process it as data.

[1744] "Image data" refers to images of the vehicle before and after repair that are taken by the user and uploaded to the cloud server.

[1745] "Emotion data" refers to data that indicates the user's emotional state as recognized by the emotion engine.

[1746] "Request ID" refers to a unique identifier generated to identify and manage a series of data related to a repair request.

[1747] "AI engine" refers to the artificial intelligence algorithm used to analyze before and after repair images and evaluate the quality of the repair.

[1748] "Judgment result" refers to the evaluation result obtained by the AI ​​engine by comparing and analyzing images before and after repair.

[1749] "Notification means" refers to a function that sends information to the user in an appropriate manner based on the judgment result and emotion data.

[1750] A "transaction" refers to a mechanism for processing a series of operations performed within a database as a single unit and maintaining its consistency.

[1751] "HTTPS protocol" refers to an internet protocol for secure communications.

[1752] MODE FOR CARRYING OUT THE INVENTION

[1753] This invention is a system that records the process in detail when a user requests car repairs, and evaluates the quality of the repair based on images taken before and after the repair and the user's emotional state. This system is composed of the user's device, a cloud server, a database, an AI engine, and an emotion engine.

[1754] Hardware and software used

[1755] Device: A smartphone or tablet operated by the user

[1756] Cloud server: A server that stores, processes, and analyzes data (e.g., AWS, Azure)

[1757] Database: A system for storing image data and emotion data (e.g., MySQL, PostgreSQL)

[1758] Emotion engine: Software for recognizing user emotions (e.g., Emotion API, IBM Watson Tone Analyzer)

[1759] AI engine: Algorithm that analyzes before and after repair images and evaluates the quality of the repair (e.g., TensorFlow, PyTorch)

[1760] System details

[1761] Before requesting car repairs, a user takes a photo of the vehicle's condition using their smartphone and uploads the image to a cloud server using a dedicated app. At this time, the emotion engine recognizes the user's emotional state (e.g., "relief") and sends that data along with the image.

[1762] The device generates a request to send the captured image data and the user's emotion data to the cloud server, and sends it to the cloud server using the HTTPS protocol.

[1763] The server stores the received images and emotion data in a database and generates a unique request ID, which is used to associate the images and emotion data before and after repair.

[1764] After the repair is complete, the user takes another photo of the vehicle's condition with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotional state (e.g., "expectation") and sends that data along with the image.

[1765] The device generates a request to send the repaired image and emotion data to the cloud server and sends it.

[1766] The server receives the post-repair images and emotion data, associates them with the request ID, and stores them in a database. It then sends the pre- and post-repair image data to the AI ​​engine, requesting it to evaluate the quality of the repair.

[1767] The AI ​​engine compares image data before and after the repair, assesses changes in the repaired area, and evaluates the quality of the repair. The results are sent back to the server and stored in a database.

[1768] The emotion engine checks the user's latest emotional state before notifying them of the evaluation results and selects an appropriate notification method (e.g., soft-spoken text).

[1769] The server generates a message to notify the user based on the judgment result and emotion data, and sends it via email or push notification.

[1770] Specific examples

[1771] For example, if a user takes a photo of the left side of a car using a smartphone and uploads the image to a cloud server using a dedicated app, the emotion engine will recognize the user's emotion as "relief" and send that data along with the image.

[1772] The device sends a request to the cloud server, and the server receives the image and emotion data and stores it in the database with request ID "12345."

[1773] After the repair is complete, the user takes another photo of the left side of the vehicle with their smartphone and uploads it to the cloud server via a dedicated app. At this time, the emotion engine again recognizes the user's emotion as "expectation" and sends the data along with it.

[1774] The device sends a request to the cloud server, and the server receives the image and emotion data, associates it with the request ID "12345," and stores it.

[1775] The server sends the image data before and after the repair to the AI ​​engine, which evaluates the quality of the repair and generates a result that the repair is valid. The server receives the result, and the emotion engine checks the user's latest emotional state and creates a soft-spoken notification message.

[1776] The server will send an email or push notification stating that the repair is legitimate, along with information that will reassure the user.

[1777] Prompt Sentence Examples

[1778] "The user takes a photo of the vehicle before repair and uploads it to the cloud server via a dedicated app. At this time, the emotion engine recognizes the user's emotional state and sends data indicating a sense of security."

[1779] The cloud server receives the image and emotion data, generates a request ID, and returns it to the user.

[1780] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1781] Step 1:

[1782] A user uses a smartphone to take a picture of the car before repair.

[1783] Input: Use the camera function of your smartphone to take a photo of the current state of the car.

[1784] Output: Vehicle exterior image data (e.g. JPEG file).

[1785] Specific operation: The user launches the app and uses the camera function to take a photo of the area to be repaired, such as the left side of the vehicle.

[1786] Step 2:

[1787] The terminal acquires image data and the user's emotional state.

[1788] Input: Image data of the vehicle and the user's facial expression captured in real time by the camera.

[1789] Output: Vehicle image data, emotion data (e.g., relief, expectation).

[1790] Specific operation: The device's emotion engine recognizes the user's face and analyzes emotions from facial expression data. For example, it uses the Emotion API to measure feelings of relief.

[1791] Step 3:

[1792] The terminal generates and transmits a request for transmitting image data and emotion data to a cloud server.

[1793] Input: Vehicle image data, emotion data, user ID, timestamp.

[1794] Output: HTTPS outgoing request to cloud server.

[1795] Specific operation: The device generates an HTTPS request and sends it to the API endpoint of the cloud server, including image data and emotion data.

[1796] Step 4:

[1797] The server stores the received image data and emotion data in a database and generates a unique request ID.

[1798] Input: Image data, emotion data, user ID, and timestamp sent to the cloud server.

[1799] Output: The database entry and the generated request ID.

[1800] Specific operation: The server starts a transaction, records the received data in the database, and simultaneously generates a request ID and returns it to the user.

[1801] Step 5:

[1802] The server returns the request ID to the user.

[1803] Input: Generated request ID.

[1804] Output: Request ID notification to the user's device.

[1805] Specific behavior: The server generates a response containing the request ID and sends it back to the user's device via HTTPS. A notification is displayed in the user's app.

[1806] Step 6:

[1807] After the repair is completed, the user takes another picture of the repaired vehicle using their smartphone.

[1808] Input: What the vehicle will look like after repairs are complete.

[1809] Output: Image data of the vehicle's exterior after repair (e.g., JPEG file).

[1810] Specific operation: The user again uses the camera function to take a picture of the condition of the vehicle after repairs.

[1811] Step 7:

[1812] The terminal acquires the image data after repair and the user's emotional state again.

[1813] Input: A captured image of the vehicle after repair, and a user's facial expression capture.

[1814] Output: Repaired image data, emotion data (e.g., expectation).

[1815] Specific operation: The device's emotion engine performs facial recognition and analyzes and generates emotion data again.

[1816] Step 8:

[1817] The terminal generates and transmits a request for transmitting the post-repair image and emotion data to a cloud server.

[1818] Input: Repaired image data, emotion data, user ID, request ID, timestamp.

[1819] Output: HTTPS outgoing request to cloud server.

[1820] Specific operation: The device generates another HTTPS request and sends the image and emotion data to the cloud server.

[1821] Step 9:

[1822] The server receives the repaired image data and emotion data, links them to the request ID, and stores them in a database.

[1823] Input: Repaired image data, emotion data, and request ID sent to the cloud server.

[1824] Output: Updated database entry.

[1825] Specific operation: The server associates the received data based on the request ID and stores it in the database. Data integrity is ensured using transactions.

[1826] Step 10:

[1827] The server sends the image data before and after the repair to the AI ​​engine and asks it to evaluate the quality of the repair.

[1828] Input: Image data before and after repair.

[1829] Output: Data analysis request to the AI ​​engine.

[1830] Specific operation: The server accesses the AI ​​engine's API, sends image data before and after repair, and requests an evaluation.

[1831] Step 11:

[1832] The AI ​​engine analyzes the image data and evaluates changes in the repair area and the quality of the repair.

[1833] Input: Image data before and after repair.

[1834] Output: Evaluation result data (e.g., "Repair is justified").

[1835] How it works: The AI ​​engine applies image analysis algorithms to evaluate changes between images to determine the quality of the repair, and generates an assessment result that is sent back to the server.

[1836] Step 12:

[1837] The server receives the evaluation results from the AI ​​engine and stores them in a database.

[1838] Input: Evaluation result data sent from the AI ​​engine.

[1839] Output: Evaluation results stored in a database.

[1840] Specific operation: The received evaluation results are associated with the request ID and stored in the database.

[1841] Step 13:

[1842] The emotion engine checks the user's latest emotional state and selects the appropriate notification method.

[1843] Input: The user's latest emotion data.

[1844] Output: Select an appropriate notification method (e.g., a soft-spoken notification).

[1845] Specific operation: The emotion engine analyzes the user's latest emotion data and determines the appropriate notification method and content.

[1846] Step 14:

[1847] The server generates a notification message based on the determination result and the selected notification method and sends it to the user.

[1848] Input: Evaluation result data, emotion data, selected notification method.

[1849] Output: A notification message to the user.

[1850] Specific operation: The server converts the judgment result into a soft-spoken sentence and sends it to the user as an email or push notification.

[1851] (Application example 2)

[1852] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1853] Conventional systems can check the vehicle's condition before and after repair using images and use AI to determine the legitimacy of the repair, but they do not take into account the user's emotional state. As a result, users often feel anxious or dissatisfied with the repair process. In particular, similar problems have occurred when determining the quality of products purchased on online shopping sites, causing a decrease in user satisfaction. The objective of this invention is to solve these problems and provide a system that can provide notifications and evaluations that take into account the user's emotions.

[1854] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a pre-repair image taken by the user and saving the image in a database, means for receiving a post-repair image taken by the user after the repair is completed and saving the image in a database, means for comparing the pre-repair and post-repair images using AI and determining the legitimacy of the repair, means for recognizing the user's emotional state and sending the emotional data along with the image to a cloud server, means for checking the user's emotional state before notification and adjusting the notification content based on that, and means for notifying the user of the determination result. This makes it possible to provide a system that takes into consideration not only repair and product quality assessment, but also the user's emotions.

[1855] "User" means an individual or corporation that uses the system to repair and assess the quality of vehicles or products.

[1856] "Image before repair" is image data showing the exterior of a vehicle or product taken by a user before requesting repair.

[1857] "Post-repair image" is image data showing the exterior of a vehicle or product taken after the user has requested repairs.

[1858] The "database" is an information management system that stores information processed and saved by the system, such as image data before and after repair and emotional data.

[1859] A "cloud server" is a remote server that processes, stores, and manages data over the Internet.

[1860] "AI" stands for artificial intelligence, a technology that uses machine learning and deep learning to analyze image data and determine the legitimacy of repairs.

[1861] An "emotion engine" is a software or hardware system for recognizing a user's emotional state and capturing that data.

[1862] "Emotion data" is data that indicates the emotional state of the user as recognized by the emotion engine.

[1863] The "judgment result" is data showing the results of the AI's analysis of pre-repair and post-repair images and its evaluation of the legitimacy of the repair.

[1864] "Notification means" refers to the method, such as email or push notification, that the system uses to notify the user of the judgment results.

[1865] In this invention, a system is implemented in which a user takes a photo of the exterior of a vehicle or product they have purchased or requested repairs on their smartphone, and then sends the image and emotional data to a cloud server. This system uses AI to analyze the images before and after the repair and evaluate the legitimacy of the repair. It also adjusts the notification content based on the user's emotional state and appropriately communicates the results to the user.

[1866] System Configuration

[1867] User terminal

[1868] Users use their smartphones to take pictures of their vehicles or products before and after repairs. The emotion engine recognizes the user's emotional state when taking the pictures and captures that data. This emotional data is then sent to a cloud server along with the images.

[1869] Cloud Server

[1870] The cloud server stores the received image data and emotional data in a database. Furthermore, the AI ​​engine compares the images before and after the repair to evaluate the legitimacy of the repair. Based on the evaluation results and the user's emotional data, the results are communicated to the user via a notification method. In particular, the system evaluates the user's emotional state before the notification and adjusts the notification content accordingly.

[1871] Hardware and software used

[1872] Smartphone: A device that allows users to take pictures, acquire emotion data, and send it to a cloud server.

[1873] OpenCV: A library for capturing images using a smartphone's camera function.

[1874] Emotion Engine: Uses emotion recognition software to analyze the user's emotional state in real time.

[1875] Cloud Server: A remote server that receives and stores data and analyzes it using AI models.

[1876] AI Engine: Artificial intelligence technology that analyzes received images and determines repair and product quality.

[1877] Notification service: A service for notifying users of results via email or push notification.

[1878] Specific examples

[1879] For example, a user orders a new smartphone from an online store and takes a picture of its packaging before receiving it. When the picture is uploaded to a cloud server via a dedicated app, the emotion engine recognizes the user's emotion as "anticipation" and sends the emotion data along with it. Later, when the user takes another picture after receiving the product and uploads it to the cloud server in the same way, the emotion engine recognizes the user's emotion as "excitement."

[1880] The cloud server sends before and after repair images to the AI ​​engine, which judges the quality of the product. As a result, it judges that "the product was delivered without any problems" and notifies the user of the judgment result using an appropriate emotional expression. At this time, the emotion engine takes into account the user's latest emotional state and notifies the user, such as "Don't worry, the product is fine."

[1881] Prompt Sentence Examples

[1882] Prompt: "Check smartphones for damage before and after delivery. Choose a flexible notification method based on user sentiment."

[1883] Model input:

[1884] Previous image: "Previous image data: <image data>"

[1885] After image: "After image data: <image data>"

[1886] Emotion Data: "Emotion Data: <emotional state>"

[1887] Model output:

[1888] Damage check: "No damage."

[1889] Notification text: "Hello, customer. No damage was found before or after delivery of the product. Please rest assured."

[1890] This invention reduces users' concerns about repairs and product quality, and allows them to receive emotionally sensitive service, which is expected to improve user satisfaction.

[1891] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1892] Step 1:

[1893] A user uses a smartphone to take a picture of a product or vehicle before repair. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain emotion data. The input is the image data captured by the smartphone camera and the emotion data recognized by the emotion engine, and the output is a set of these data.

[1894] Step 2:

[1895] The device sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format and the emotion data is packaged in JSON format. The input is the captured image data and emotion data, and the output is a transmission request to the cloud server.

[1896] Step 3:

[1897] The cloud server stores the received image data and emotion data in a database. When storing, it generates a unique request ID and manages it together with the image and emotion data. The input is the image data and emotion data sent from the device, and the output is the data stored in the database and the generated request ID.

[1898] Step 4:

[1899] After the repair or receiving of the product, the user uses their smartphone to take a picture of the product after repair or receiving it. At this time, the emotion engine again recognizes the user's emotion and acquires emotion data. The input is the image data captured by the smartphone camera and the recognized emotion data, and the output is a set of these data.

[1900] Step 5:

[1901] The device then sends the captured image and emotion data to the cloud server. During transmission, the image data is converted to JPEG format again, and the emotion data is packaged in JSON format. The input is the captured image data and emotion data after repair, and the output is a transmission request to the cloud server.

[1902] Step 6:

[1903] The cloud server stores the received image data and emotion data in a database and associates them with the existing request ID. The input is the repaired image data and emotion data sent from the device, and the output is the data stored in the database.

[1904] Step 7:

[1905] The cloud server sends the image data before and after the repair to the AI ​​engine to evaluate the legitimacy of the repair. The input is the image data before and after the repair stored in the database, and the output is the analysis result of the AI ​​engine.

[1906] Step 8:

[1907] The AI ​​engine compares images before and after repair to evaluate the quality and legitimacy of the repair. Specifically, it uses an image analysis algorithm to determine changes in the repair area and the accuracy of the repair. The input is image data before and after the repair, and the output is an evaluation result regarding the legitimacy of the repair.

[1908] Step 9:

[1909] The cloud server receives the evaluation results and stores them in a database. Furthermore, the emotion engine determines the user's emotional state before the notification and adjusts the notification content accordingly. The input is the AI ​​engine's evaluation results and the user's latest emotional data, and the output is the information notified to the user.

[1910] Step 10:

[1911] The cloud server sends the evaluation results to the user via email or push notification via the notification means. The notification content is appropriately adjusted based on the user's emotional state. The input is the notification information sent from the cloud server, and the output is the notification content displayed on the user's device.

[1912] This system makes it possible to provide advanced services that take into consideration the user's emotions during the repair and product quality assessment process.

[1913] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1914] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1915] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1916] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1917] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1918] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1919] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1920] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1921] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1922] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1923] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1924] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1925] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1926] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1927] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1928] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1929] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1930] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1931] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1932] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1933] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1934] The following is further disclosed regarding the above embodiment.

[1935] (Claim 1)

[1936] means for receiving an image of the product before repair taken by a user and storing the image in a database;

[1937] a means for receiving a post-repair image taken by a user after the repair is completed and storing the image in a database;

[1938] A means of using AI to compare before and after repair images and determine the legitimacy of the repair;

[1939] The system includes a means for notifying the user of the determination result.

[1940] (Claim 2)

[1941] 2. The system according to claim 1, further comprising means for storing the determination result in a database and for evaluating the repairer based on the determination result.

[1942] (Claim 3)

[1943] 2. The system according to claim 1, further comprising means for sending the determination result to the user by email or push notification.

[1944]

[1945] "Example 1"

[1946] (Claim 1)

[1947] means for receiving an image of the product before repair taken by a user and storing the image in a storage device;

[1948] a means for receiving an image of the repaired product taken by the user after the repair is completed and storing the image in a storage device;

[1949] A means for analyzing before-repair and after-repair images using a generative AI model to determine the legitimacy of the repair;

[1950] The system includes a means for notifying the user of the determination result.

[1951] (Claim 2)

[1952] 2. The system according to claim 1, further comprising means for storing the determination result in a database and for evaluating the repairer based on the determination result.

[1953] (Claim 3)

[1954] 2. The system according to claim 1, further comprising means for transmitting the determination result to the user via communication means.

[1955] "Application Example 1"

[1956] (Claim 1)

[1957] means for receiving an image of the product before repair taken by a user and storing the image in a database;

[1958] a means for receiving a post-repair image taken by a user after the repair is completed and storing the image in a database;

[1959] A means of using AI to compare before and after repair images and determine the legitimacy of the repair;

[1960] means for notifying a user of the determination result;

[1961] a means for automatically photographing the exterior of the vehicle using a photographing device mounted on the vehicle;

[1962] A means for uploading the image data to a cloud server;

[1963] A means for the cloud server to generate a unique request ID;

[1964] A means for storing the captured image in a database based on the generated request ID;

[1965] A

[1966] A system including:

[1967] (Claim 2)

[1968] 2. The system according to claim 1, further comprising means for storing the determination result in a database and for evaluating the repairer based on the determination result.

[1969] (Claim 3)

[1970] 2. The system according to claim 1, further comprising means for sending the determination result to the user by email or push notification.

[1971] "Example 2: Combining Emotion Engines"

[1972] (Claim 1)

[1973] means for receiving an image of the product before repair taken by a user and storing the image in a database;

[1974] means for recognizing the emotional state of the user and transmitting the emotional data together with the image;

[1975] a means for receiving a post-repair image taken by a user after the repair is completed and storing the image in a database;

[1976] A means of using AI to compare before and after repair images and determine the legitimacy of the repair;

[1977] means for selecting an appropriate notification method based on the determination result and the user's latest emotional state, and notifying the user;

[1978] A system including:

[1979] (Claim 2)

[1980] 2. The system according to claim 1, further comprising means for storing the judgment result and the emotion data in a database and for evaluating the repairer ...

Claims

1. means for receiving an image of the product before repair taken by a user and storing the image in a database; a means for receiving a post-repair image taken by a user after the repair is completed and storing the image in a database; A means of using AI to compare before and after repair images and determine the legitimacy of the repair; The system includes a means for notifying the user of the determination result.

2. 2. The system according to claim 1, further comprising means for storing the determination result in a database and for evaluating the repairer based on the determination result.

3. The system according to claim 1 , further comprising means for sending the determination result to the user by email or push notification.

Images