system

Abstract

We provide the system. [Solution] A means of obtaining return information after use, A means for photographing the exterior of the terminal based on the aforementioned return information, A means for collecting the captured image data via a communication network, A means of analyzing the accumulated image data using an analysis model, A means for comparing the results of the analysis model with past data records to determine damage, A means of reporting the status of the returned device based on the judgment result, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The determination of damage at the time of returning the terminal has been conventionally performed manually, and the lack of uniformity in determination and the delay in processing have become problems. Therefore, it is necessary to realize a quick and consistent damage determination and improve the efficiency of the process for determining the reimbursement amount.

Means for Solving the Problems

[0005] The present invention provides means for acquiring return information and photographing the appearance of the terminal. Further, the present invention has means for collecting the photographed image data, analyzing it using an analysis model, comparing the past data records and results, and determining damage. Furthermore, based on this determination result, the present invention provides a system for reporting the state of the terminal, automatically determining and notifying a reimbursement request as necessary.

[0006] "Return information after use" refers to information obtained when a user returns a device they have used to a designated location.

[0007] "Means for photographing the external appearance" refers to a process or apparatus that uses a camera or other photographic device to visually record the physical state of a terminal.

[0008] "Means of aggregating data via a communication network" refers to a process or mechanism for collecting and aggregating data sent from terminals via the internet or dedicated lines.

[0009] An "analysis model" is an algorithm or machine learning system designed to analyze image data from a device and identify damage or anomalies.

[0010] "Data recording" refers to an information aggregate that stores and reuses information, including past device damage and usage history.

[0011] "Means for determining damage" refers to a process or system for determining whether and to what extent physical damage has occurred to the terminal based on the analysis results.

[0012] "Means for reporting the condition of returned devices based on the assessment results" refers to methods or devices for communicating the damage assessment results to the relevant parties.

[0013] "A means of automatically determining whether or not to demand payment and notifying the user" refers to a mechanism that uses the judgment result to automatically determine whether or not the user needs to bear the costs and notifies the user of that decision. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0018] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0019] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The present invention provides a system that automatically determines the extent of damage to a terminal during the terminal return process. The system includes a server, terminals, a communication network, and an AI-based analysis model.

[0036] First, the user returns the device after use. Once the device is returned, the built-in camera takes pictures of the device's exterior from multiple angles. These images are acquired in high resolution and temporarily stored in the device's storage. The device then sends the captured images to the server via the communication network.

[0037] The server inputs the received image data into an AI analysis model. This analysis model uses deep learning technology and analyzes the images based on various damage patterns it has learned in the past. During feature extraction, it is designed to identify damage patterns such as scratches, cracks, and discoloration. The analysis results are compared with past damage cases recorded in the server's database.

[0038] Once the server finishes its assessment, a result is generated. Based on this result, it determines whether the returned device was intentionally damaged or whether the damage was due to normal use. Through this automated assessment, the server generates a report and notifies the user.

[0039] As a concrete example, consider a case where a cracked screen is found on a user's returned device. The server analyzes the image taken by the device, and the AI ​​model recognizes it as intentional damage. If it is determined to be new damage through comparison with past data, the server will determine that compensation is required and record this in the report. The user will then be notified and provided with information about the repair costs.

[0040] This system automates the previously reliant manual damage assessment process, enabling faster and more accurate determinations.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user returns the device. The system's return process begins when the user returns the device to the designated location after use.

[0044] Step 2:

[0045] The device takes pictures of its exterior. Upon receiving a returned device, it uses its built-in camera to take high-resolution images of its exterior from multiple angles. The captured images are temporarily stored on the device.

[0046] Step 3:

[0047] The device sends the image to the server. The captured image data is compressed via the communication network and securely uploaded to the server.

[0048] Step 4:

[0049] The server analyzes the images. The server inputs the received images into an AI analysis model, which uses deep learning techniques to extract damage characteristics. This model is trained to identify patterns of damage and failure.

[0050] Step 5:

[0051] The server compares the current situation to past database data. Based on the features extracted by the AI ​​model, the server compares the current situation to past damage cases in the database. It calculates similarity and determines whether the damage is existing wear and tear or new damage.

[0052] Step 6:

[0053] The server determines the result and generates a report. Based on the determination, the server generates a report detailing the necessary repayments if any are required.

[0054] Step 7:

[0055] The server notifies the user. Based on the report, the user is notified of the result of the device return and, if necessary, is provided with information regarding compensation claims.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In recent years, as the use of information terminals has expanded, there has been a demand for faster and fairer damage assessments upon terminal return. However, manual assessments are subjective and can lack fairness. Furthermore, while rapid response is required, efficient methods for assessing terminal condition have not yet been sufficiently established. Therefore, there is a need for automated technology that can fairly and quickly assess terminal damage and take appropriate measures.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for acquiring information regarding the return of an information terminal, means for capturing images of the information terminal's appearance from multiple viewpoints, means for accumulating image data via an information communication network, means for analyzing data using an analysis model, and means for notifying the user of the determination results. This makes it possible to automatically and quickly evaluate the damage status of the terminal and take fair and appropriate measures for the user.

[0061] "Information regarding the return of an information terminal after use" refers to a series of data and information recorded when a user returns an information terminal after using it.

[0062] An "information terminal" is an electronic device that a user can operate and use, and includes devices such as computers, smartphones, and tablets.

[0063] "Imaging means" refers to a device or method used to photograph the external appearance of an information terminal, and utilizes a device such as a camera.

[0064] An "information and communication network" refers to communication infrastructure such as the internet and mobile networks used to send and receive data.

[0065] An "analytical model" is an algorithm or software used to analyze given data, and includes techniques such as machine learning and deep learning.

[0066] The "judgment result" is the result of the analysis obtained by the analytical model, and indicates a judgment regarding the damage status of the information terminal.

[0067] A "claim for damages" is the act of claiming necessary expenses or compensation based on whether or not the returned information terminal was damaged and to what extent.

[0068] The system of the present invention is for automating the assessment of damage to information terminals upon their return. Specifically, it comprises an information terminal, a server, a communication network, and a generative AI model.

[0069] After use, the user places the information terminal in a designated return location. The terminal automatically uses its built-in camera to capture images of its exterior from multiple angles. High-resolution image data is temporarily stored on the terminal. The stored image data is then transmitted to a server via a communication network. Wi-Fi or a mobile network is used during this process, and the transmitted image data is encrypted.

[0070] The server inputs the received image data into an analysis model that uses machine learning techniques. This analysis model is based on common deep learning frameworks such as TENSORFLOW® and PyTorch, and analyzes the images using a variety of damage patterns that it has learned in the past. It can automatically extract characteristic damage patterns, such as scratches, cracks, and discoloration.

[0071] The analysis results are compared with historical information stored in the server's database to determine whether the damage was intentional or due to normal use. A report is then automatically generated and sent to the user via email. The report details the damage and necessary actions (e.g., repair costs).

[0072] Example prompt: "Analyze the images of the returned device to determine if there is any damage and what the cause is."

[0073] This system enables a rapid and fair assessment of the device's damage status and allows for appropriate countermeasures to be taken.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] The user returns the information terminal to a designated location. The terminal automatically activates its camera and takes pictures of its exterior from various angles. The input is external information of the terminal, and the output is high-resolution image data. This operation collects data to understand the terminal's status from all angles.

[0077] Step 2:

[0078] The device temporarily stores captured image data in its internal storage. Simultaneously, once a network connection is established, it transmits the image data to the server via the communication network. The input is the data stored on the device, and the output is the received data on the server. Data encryption protocols are used during communication to ensure security.

[0079] Step 3:

[0080] The server inputs the received image data into a machine learning model and begins the analysis. The model uses a framework such as TensorFlow to extract features based on previously trained datasets. The input is image data, and the output is detection information regarding damage such as scratches, cracks, and discoloration. This process automatically evaluates the device's damage status.

[0081] Step 4:

[0082] The server identifies the cause of the damage by comparing the analysis results with past cases in its internal database. The inputs are the analysis results and past case data, while the output is damage assessment and judgment information. By identifying similar cases and patterns, it determines whether the damage was intentional.

[0083] Step 5:

[0084] The server generates a report based on the assessment information and notifies the user via email. The report includes details of the damage, the reason for the assessment, and the necessary procedures. The input is the damage assessment information, and the output is the generated report. By quickly delivering this information to the user, the server enables appropriate action.

[0085] (Application Example 1)

[0086] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] In logistics centers and other inventory management facilities, the inspection of goods during package receipt and shipment relies on manual processes, making judgment criteria subjective and hindering quick and accurate damage assessment. Furthermore, there is a need to automate the process for making fair judgments regarding damage and the need for compensation.

[0088] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0089] In this invention, the server includes means for acquiring return information after use, means for photographing the exterior based on the return information, means for accumulating the captured image data via a communication infrastructure, means for analyzing the accumulated image data using an analysis model, means for comparing the results of the analysis model with past data records to determine damage, means for reporting the condition of the returned equipment based on the determination result, and means for determining compensation based on the damage result and creating a report for the person in charge. This enables rapid and objective determination of package damage and automation of fair compensation decisions.

[0090] "Return information" refers to the data associated with a used item or device being returned to the administrator, including recorded history and status.

[0091] "Exterior" refers to the parts that make up the outside of an item or piece of equipment, and describes its appearance and physical condition.

[0092] A "communication infrastructure" is a network infrastructure used to send and receive digital data, and is a system that can transmit information quickly.

[0093] An "analytical model" is a mathematical or algorithmic structure used to analyze collected data and extract meaningful information.

[0094] "Damage assessment" is the process of determining whether an item or piece of equipment is in good condition or damaged.

[0095] A "report" is a written or digital document that summarizes analysis and judgment results and systematically describes related information.

[0096] The system for realizing this invention will be used in inventory management facilities such as logistics centers. When used items are returned to the facility, a terminal is used to photograph the exterior. The terminal is a smartphone or tablet equipped with a high-resolution camera, and it photographs the exterior of the item from multiple angles. The captured image data is temporarily stored in the terminal and then transmitted to a server using a communication infrastructure.

[0097] The server processes the received image data and analyzes it using an analysis model based on machine learning algorithms. This analysis utilizes deep learning frameworks such as TensorFlow and Python to extract damage features. The analysis results are compared with historical data records to determine the extent of the damage. Based on the determination, a report is automatically generated and notified to the responsible party. The generated report is properly stored and managed in a MySQL® database.

[0098] As a concrete example, when a package of confectionery arrives at the distribution center, the terminal takes a picture of the package, and the data is analyzed on a server. The AI ​​model identifies a new dent in the box, and the report states, "Damage found. Compensation required." The results are quickly notified to the person in charge through the application.

[0099] An example of a prompt statement to be input into the generating AI model is, "Analyze images of packages in transit at the logistics center and identify any damage." This prompt statement helps to make more accurate damage detections.

[0100] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0101] Step 1:

[0102] The device takes high-resolution photographs of the exterior of used items from multiple angles. The input is the actual item, and the output is the captured image data. The captured image data is temporarily stored within the device.

[0103] Step 2:

[0104] The device transmits the captured image data to the server using the communication infrastructure. The input is the stored image data, and the output is the completion of the transfer of the image data to the server. This process prepares the data for analysis.

[0105] Step 3:

[0106] The server inputs the received image data into an analysis model using TensorFlow and performs image analysis. The input is the transmitted image data, and the output is the analysis results regarding the damage features. Here, feature extraction is performed using deep learning techniques.

[0107] Step 4:

[0108] The server compares the analysis results with historical data records and a MySQL database to determine if there is damage. The input consists of the analysis results and historical data records, and the output is a determination of whether or not there is damage. This determination determines whether the item is normal or damaged.

[0109] Step 5:

[0110] The server automatically generates a report based on the assessment result and notifies the responsible person. The input is the assessment result, and the output is the generated report and the notification. The report includes details of the damage and whether compensation is required, and is designed to reach the responsible person quickly.

[0111] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0112] This invention relates to a system that integrates user emotion recognition technology into the damage assessment process when a device is returned. The system includes a device, a server, a communication network, an image analysis model, and an emotion engine.

[0113] When a user returns a device, the device's camera captures a detailed image of the device's appearance upon return. This image data is transmitted to a server via the network. Simultaneously, an emotion engine built into the device analyzes the user's facial expressions and voice, and sends the results to the server. The emotion engine has the capability to accurately determine whether the user is nervous or relieved upon returning the device.

[0114] The server analyzes the received image data using a modifiable deep learning model. The deep learning model identifies damage characteristics based on a vast amount of historical data and compares them to previous damage records. User sentiment data can also be used in the analysis, allowing the judgment method to be adjusted according to the user's state.

[0115] Based on the analysis results, the server determines the extent of damage to the returned device and generates a report. This report is presented in an appropriate format, taking into account the user's emotional state. For example, if the user is stressed when returning the device, the notification content is adjusted to be gentler and more reassuring.

[0116] As a concrete example, consider a case where a user returns a device with a small crack on the screen. If the emotion engine analyzes that the user is showing anxiety or tension regarding this situation, the server will send a notification that includes a sincere response in addition to the usual repair procedures. This will reassure the user, and the problem will be resolved.

[0117] This system not only streamlines the process of determining damage when devices are returned, but also improves the user experience.

[0118] The following describes the processing flow.

[0119] Step 1:

[0120] The user returns the device. The return system is activated when the user acknowledges that they have finished using the device and brings it to the designated return location.

[0121] Step 2:

[0122] The device takes pictures of its exterior. Upon receiving the device, it uses its built-in camera to take pictures of its exterior from multiple angles. During this process, the device automatically checks the image quality and retakes the pictures if necessary.

[0123] Step 3:

[0124] The device activates the emotion engine. The emotion engine records the user's facial expressions and voice, and analyzes the user's emotional state. For example, it can determine in real time whether the user is smiling or feeling nervous.

[0125] Step 4:

[0126] The device sends images and emotion data captured by the device to the server. Through the communication network, the device uploads high-resolution image data along with the analyzed user emotion data to the server.

[0127] Step 5:

[0128] The server analyzes the image data. An AI analysis model installed on the server uses deep learning technology to analyze the images and extract the characteristics of the damage that occurred on the device.

[0129] Step 6:

[0130] The server incorporates emotional data into its analysis. The server uses the user's emotional data to adjust the damage assessment and make more appropriate judgments.

[0131] Step 7:

[0132] The server performs damage assessment and generates a report. Based on the analysis results, it determines whether the terminal's condition is due to intentional damage or normal wear and tear, and generates a detailed report.

[0133] Step 8:

[0134] The server notifies the user. Considering the report's contents, it creates an appropriate notification message tailored to the user's emotional state and sends it to the user via email, app notifications, or other means.

[0135] (Example 2)

[0136] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0137] Conventional equipment return damage assessment processes merely assessed and reported only the physical damage to the equipment, lacking consideration for the user's psychological state. As a result, users were not provided with adequate reassurance or support, sometimes leading to a poor user experience. This invention aims to improve the user experience by considering the user's emotions during the return process and providing more appropriate assessment and reporting.

[0138] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0139] In this invention, the server includes means for acquiring return information after use and user emotional information, means for analyzing the accumulated image data and emotional information, and means for reporting the damage status of the equipment based on the analysis results and notifying the user in a format that takes into account their emotional state. This enables comprehensive judgment and reporting that takes into account not only physical damage but also the user's psychological state.

[0140] "Return information" refers to data acquired when a user returns equipment, and it indicates the status of the return and its physical condition.

[0141] "Emotional information" refers to data that indicates the user's psychological state, extracted from their facial expressions and voice, and is used to identify the user's emotions at the time of return.

[0142] A "communication network" is a system that exchanges data via various wired or wireless technologies, using paths and media for transmitting information.

[0143] An "analysis algorithm" is a procedure or method for processing large amounts of data and extracting or determining information according to a specific purpose.

[0144] A "machine learning algorithm" is a set of methods that automatically learn from past data and experience to perform identification and prediction, and is used to recognize patterns in data.

[0145] "Damage assessment" refers to the process of identifying and evaluating whether there is any physical or functional damage to equipment, and to what extent.

[0146] "Notification" refers to the act of conveying a decision or information to a user, and involves sending an appropriate message depending on the situation.

[0147] This system is an integrated system designed to take into account the user's emotions when equipment is returned, enabling more appropriate damage assessment and reporting. It mainly consists of a combination of terminals, servers, communication networks, data analysis algorithms, and an emotion analysis engine.

[0148] The device automatically activates its camera when the user returns the equipment, taking a picture of the equipment's appearance upon return. In addition, it records the user's emotional information using facial recognition and voice analysis technologies. This includes common computer vision techniques for face detection and software for voice analysis. This information is transmitted to a server using wireless communication technologies (e.g., Wi-Fi or mobile networks).

[0149] The server processes the received image data and sentiment data. The image data is analyzed using machine learning algorithms to identify corruption features. It is preferable to use deep learning frameworks such as TensorFlow or PyTorch for the analysis algorithms. The analysis results are compared with historical databases, and if new corruption is detected, its details are reflected in the report.

[0150] Emotional data is used to determine the user's psychological state. The emotion analysis engine analyzes the emotions the user is experiencing at the time of return based on collected facial and voice data. The resulting emotional state is used to adjust the content and tone of the report. For example, if the user is feeling anxious, the report will be written in a more considerate manner, and additional reassuring information will be provided.

[0151] As a concrete example, consider a case where a user returns a smartphone with a small crack in the screen. If the emotion analysis engine determines that the user appears anxious, the server will generate a reassuring notification in addition to the usual damage assessment. This allows the user to feel that they are receiving a sincere response and to have confidence that the problem will be resolved.

[0152] Example of a prompt:

[0153] "Please explain how you will analyze the emotions users experience when returning their devices and how you will adjust the damage assessment process based on those results."

[0154] This system enables efficient and highly satisfying service delivery through a comprehensive approach that considers not only physical damage but also the user's psychological state.

[0155] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0156] Step 1:

[0157] The device automatically activates its built-in camera upon return and takes a picture of the device's exterior. The captured image data is then processed, including exposure correction and noise reduction, before being sent to the server. The input is the physical exterior of the device, and the output is the processed image data.

[0158] Step 2:

[0159] The device records the user's facial expressions and voice, and analyzes them in real time using an emotion analysis engine. This analysis utilizes facial recognition technology and voice feature analysis to identify the user's emotional state (e.g., tension, relief, anxiety). Input consists of the user's facial image and voice data, and output is the determined emotion information.

[0160] Step 3:

[0161] The server feeds the received image data into a machine learning algorithm to extract damage features. This process uses a conventional deep learning model and compares the data against a historical database. The input is image data, and the output is information indicating the location and characteristics of the damage.

[0162] Step 4:

[0163] The server analyzes emotional information and evaluates the user's psychological state. Based on this information, it adjusts the criteria for damage assessment to perform a more accurate evaluation. The input is emotional information, and the output is the adjusted damage assessment criteria.

[0164] Step 5:

[0165] The server generates a report on the equipment's damage status based on the integrated analysis results. Since the content and tone of this report are adjusted according to the user's emotional state, the inputs are damage information and emotional information, and the output is a customized report for the user.

[0166] (Application Example 2)

[0167] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0168] There is a need for a system that simultaneously improves damage assessment and user experience when returning consumer robots. In particular, the challenge is to enable flexible responses that take into account the user's emotions and reduce stress during the return process.

[0169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0170] In this invention, the server includes means for acquiring return information after use, means for photographing the appearance of the equipment, means for accumulating the captured image data via a communication network, means for analyzing the accumulated image data using an analysis model, means for analyzing the user's emotions, and means for adaptively adjusting the report content based on the results of the emotion analysis means. This not only enables efficient determination of equipment damage upon return but also allows for flexible responses in accordance with the user's psychological state.

[0171] "Means for obtaining return information after use" refers to a device or method that has the function of obtaining information regarding the return of equipment when its use has ended.

[0172] "Means for photographing the appearance of equipment" refers to a device or method for photographing the appearance of equipment as an image in order to record the surface condition and characteristics of the equipment.

[0173] "Means of aggregating information via a communication network" refers to a device or method that has the function of collecting information through a network into a centralized database or server.

[0174] An "analysis model" is a computational model built based on past data and learning results in order to analyze acquired data.

[0175] "Emotional analysis means" refers to a device or method for detecting and analyzing emotions from a user's facial expressions, voice, etc.

[0176] "Means of adaptive adjustment" refers to a device or method that has the function of automatically changing the optimal response according to the situation based on the analysis results.

[0177] The system for carrying out this invention mainly includes a server, a terminal, a communication network, a camera, and an emotion analysis engine. The terminal is equipped with a camera that acquires return information after use and also takes pictures of the device's appearance. The captured image data is transmitted to the server via the communication network.

[0178] After receiving this image data, the server analyzes the data using an analysis model. This analysis model is built on a deep learning algorithm and determines equipment damage by comparing past data records with current image data.

[0179] Furthermore, the emotion analysis engine installed in the device analyzes the user's emotions from their facial expressions and voice. This emotional data is sent to a server and reflected in the damage assessment report. Based on this information, the server adaptively adjusts the report content according to the user's emotions and notifies the user. This notification is designed to be flexible and considerate using a generative AI model.

[0180] For example, if a user returns equipment and notices minor cosmetic damage, and the user expresses tension or anxiety, the system will generate a notification that uses gentle language to explain the situation and provide reassurance.

[0181] An example of a prompt message is, "Generate thoughtful feedback based on the user's feelings and the device's condition upon return."

[0182] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0183] Step 1:

[0184] The terminal retrieves return information and uses its camera to photograph the exterior of the returned equipment. The input is the returned equipment, and the output is an image of the equipment's exterior. The terminal uses its camera to take detailed photographs of the entire front surface of the equipment.

[0185] Step 2:

[0186] The device transmits image data it captures to a server via a communication network. The input is an image of the device's exterior, and the output is the image data transmitted to the server. The device performs appropriate compression and encryption to securely upload the data to the server.

[0187] Step 3:

[0188] The server inputs the received image data into an analysis model and analyzes the data. The input is the image data sent to the server, and the output is a judgment result indicating the presence and extent of damage. The server utilizes a deep learning algorithm to identify damage characteristics by comparing them with past data.

[0189] Step 4:

[0190] The device detects the user's facial expressions and voice, and analyzes those emotions using an emotion analysis engine. The input is the user's facial image and voice data, and the output is data indicating the user's emotional state. The device performs emotion analysis in real time, determining things like tension and a sense of security.

[0191] Step 5:

[0192] The server receives emotion analysis data and generates a damage assessment report based on it. The input is the damage assessment result and the user's emotion data, and the output is the report content to be notified to the user. The server uses a generation AI model and utilizes prompt text to generate feedback content.

[0193] Step 6:

[0194] The user receives the report and chooses further actions based on it. The input is the report information notified to the user, and the output is the user's next action or feedback. The user can proceed with the return process with confidence.

[0195] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0196] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0197] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0198] [Second Embodiment]

[0199] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0200] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0201] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0202] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0203] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0204] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0205] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0206] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0207] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0208] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0209] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0210] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0211] The present invention provides a system that automatically determines the extent of damage to a terminal during the terminal return process. The system includes a server, terminals, a communication network, and an AI-based analysis model.

[0212] First, the user returns the device after use. Once the device is returned, the built-in camera takes pictures of the device's exterior from multiple angles. These images are acquired in high resolution and temporarily stored in the device's storage. The device then sends the captured images to the server via the communication network.

[0213] The server inputs the received image data into an AI analysis model. This analysis model uses deep learning technology and analyzes the images based on various damage patterns it has learned in the past. During feature extraction, it is designed to identify damage patterns such as scratches, cracks, and discoloration. The analysis results are compared with past damage cases recorded in the server's database.

[0214] Once the server finishes its assessment, a result is generated. Based on this result, it determines whether the returned device was intentionally damaged or whether the damage was due to normal use. Through this automated assessment, the server generates a report and notifies the user.

[0215] As a concrete example, consider a case where a cracked screen is found on a user's returned device. The server analyzes the image taken by the device, and the AI ​​model recognizes it as intentional damage. If it is determined to be new damage through comparison with past data, the server will determine that compensation is required and record this in the report. The user will then be notified and provided with information about the repair costs.

[0216] This system automates the previously reliant manual damage assessment process, enabling faster and more accurate determinations.

[0217] The following describes the processing flow.

[0218] Step 1:

[0219] The user returns the device. The system's return process begins when the user returns the device to the designated location after use.

[0220] Step 2:

[0221] The device takes pictures of its exterior. Upon receiving a returned device, it uses its built-in camera to take high-resolution images of its exterior from multiple angles. The captured images are temporarily stored on the device.

[0222] Step 3:

[0223] The device sends the image to the server. The captured image data is compressed via the communication network and securely uploaded to the server.

[0224] Step 4:

[0225] The server analyzes the images. The server inputs the received images into an AI analysis model, which uses deep learning techniques to extract damage characteristics. This model is trained to identify patterns of damage and failure.

[0226] Step 5:

[0227] The server compares the current situation to past database data. Based on the features extracted by the AI ​​model, the server compares the current situation to past damage cases in the database. It calculates similarity and determines whether the damage is existing wear and tear or new damage.

[0228] Step 6:

[0229] The server determines the result and generates a report. Based on the determination, the server generates a report detailing the necessary repayments if any are required.

[0230] Step 7:

[0231] The server notifies the user. Based on the report, the user is notified of the result of the device return and, if necessary, is provided with information regarding compensation claims.

[0232] (Example 1)

[0233] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0234] In recent years, as the use of information terminals has expanded, there has been a demand for faster and fairer damage assessments upon terminal return. However, manual assessments are subjective and can lack fairness. Furthermore, while rapid response is required, efficient methods for assessing terminal condition have not yet been sufficiently established. Therefore, there is a need for automated technology that can fairly and quickly assess terminal damage and take appropriate measures.

[0235] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0236] In this invention, the server includes means for acquiring information regarding the return of an information terminal, means for capturing images of the information terminal's appearance from multiple viewpoints, means for accumulating image data via an information communication network, means for analyzing data using an analysis model, and means for notifying the user of the determination results. This makes it possible to automatically and quickly evaluate the damage status of the terminal and take fair and appropriate measures for the user.

[0237] "Information regarding the return of an information terminal after use" refers to a series of data and information recorded when a user returns an information terminal after using it.

[0238] An "information terminal" is an electronic device that a user can operate and use, and includes devices such as computers, smartphones, and tablets.

[0239] "Imaging means" refers to a device or method used to photograph the external appearance of an information terminal, and utilizes a device such as a camera.

[0240] An "information and communication network" refers to communication infrastructure such as the internet and mobile networks used to send and receive data.

[0241] An "analytical model" is an algorithm or software used to analyze given data, and includes techniques such as machine learning and deep learning.

[0242] The "judgment result" is the result of the analysis obtained by the analytical model, and indicates a judgment regarding the damage status of the information terminal.

[0243] A "claim for damages" is the act of claiming necessary expenses or compensation based on whether or not the returned information terminal was damaged and to what extent.

[0244] The system of the present invention is for automating the assessment of damage to information terminals upon their return. Specifically, it comprises an information terminal, a server, a communication network, and a generative AI model.

[0245] After use, the user places the information terminal in a designated return location. The terminal automatically uses its built-in camera to capture images of its exterior from multiple angles. High-resolution image data is temporarily stored on the terminal. The stored image data is then transmitted to a server via a communication network. Wi-Fi or a mobile network is used during this process, and the transmitted image data is encrypted.

[0246] The server inputs the received image data into an analysis model that uses machine learning techniques. This analysis model is based on common deep learning frameworks such as TensorFlow and PyTorch, and analyzes the images using a variety of damage patterns that it has learned in the past. It can automatically extract characteristic damage patterns, such as scratches, cracks, and discoloration.

[0247] The analysis results are compared with historical information stored in the server's database to determine whether the damage was intentional or due to normal use. A report is then automatically generated and sent to the user via email. The report details the damage and necessary actions (e.g., repair costs).

[0248] Example prompt: "Analyze the images of the returned device to determine if there is any damage and what the cause is."

[0249] This system enables a rapid and fair assessment of the device's damage status and allows for appropriate countermeasures to be taken.

[0250] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0251] Step 1:

[0252] The user returns the information terminal to a designated location. The terminal automatically activates its camera and takes pictures of its exterior from various angles. The input is external information of the terminal, and the output is high-resolution image data. This operation collects data to understand the terminal's status from all angles.

[0253] Step 2:

[0254] The device temporarily stores captured image data in its internal storage. Simultaneously, once a network connection is established, it transmits the image data to the server via the communication network. The input is the data stored on the device, and the output is the received data on the server. Data encryption protocols are used during communication to ensure security.

[0255] Step 3:

[0256] The server inputs the received image data into a machine learning model and begins the analysis. The model uses a framework such as TensorFlow to extract features based on previously trained datasets. The input is image data, and the output is detection information regarding damage such as scratches, cracks, and discoloration. This process automatically evaluates the device's damage status.

[0257] Step 4:

[0258] The server identifies the cause of the damage by comparing the analysis results with past cases in its internal database. The inputs are the analysis results and past case data, while the output is damage assessment and judgment information. By identifying similar cases and patterns, it determines whether the damage was intentional.

[0259] Step 5:

[0260] The server generates a report based on the assessment information and notifies the user via email. The report includes details of the damage, the reason for the assessment, and the necessary procedures. The input is the damage assessment information, and the output is the generated report. By quickly delivering this information to the user, the server enables appropriate action.

[0261] (Application Example 1)

[0262] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0263] In logistics centers and other inventory management facilities, the inspection of goods during package receipt and shipment relies on manual processes, making judgment criteria subjective and hindering quick and accurate damage assessment. Furthermore, there is a need to automate the process for making fair judgments regarding damage and the need for compensation.

[0264] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0265] In this invention, the server includes means for acquiring return information after use, means for photographing the exterior based on the return information, means for accumulating the captured image data via a communication infrastructure, means for analyzing the accumulated image data using an analysis model, means for comparing the results of the analysis model with past data records to determine damage, means for reporting the condition of the returned equipment based on the determination result, and means for determining compensation based on the damage result and creating a report for the person in charge. This enables rapid and objective determination of package damage and automation of fair compensation decisions.

[0266] "Return information" refers to the data associated with a used item or device being returned to the administrator, including recorded history and status.

[0267] "Exterior" refers to the parts that make up the outside of an item or piece of equipment, and describes its appearance and physical condition.

[0268] A "communication infrastructure" is a network infrastructure used to send and receive digital data, and is a system that can transmit information quickly.

[0269] An "analytical model" is a mathematical or algorithmic structure used to analyze collected data and extract meaningful information.

[0270] "Damage assessment" is the process of determining whether an item or piece of equipment is in good condition or damaged.

[0271] A "report" is a written or digital document that summarizes analysis and judgment results and systematically describes related information.

[0272] The system for realizing this invention will be used in inventory management facilities such as logistics centers. When used items are returned to the facility, a terminal is used to photograph the exterior. The terminal is a smartphone or tablet equipped with a high-resolution camera, and it photographs the exterior of the item from multiple angles. The captured image data is temporarily stored in the terminal and then transmitted to a server using a communication infrastructure.

[0273] The server processes the received image data and analyzes it using an analysis model based on machine learning algorithms. This analysis utilizes deep learning frameworks such as TensorFlow and Python to extract damage features. The analysis results are compared with historical data records to determine the extent of the damage. Based on the determination, a report is automatically generated and notified to the responsible party. The generated report is properly stored and managed in a MySQL database.

[0274] As a concrete example, when a package of confectionery arrives at the distribution center, the terminal takes a picture of the package, and the data is analyzed on a server. The AI ​​model identifies a new dent in the box, and the report states, "Damage found. Compensation required." The results are quickly notified to the person in charge through the application.

[0275] An example of a prompt statement to be input into the generating AI model is, "Analyze images of packages in transit at the logistics center and identify any damage." This prompt statement helps to make more accurate damage detections.

[0276] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0277] Step 1:

[0278] The device takes high-resolution photographs of the exterior of used items from multiple angles. The input is the actual item, and the output is the captured image data. The captured image data is temporarily stored within the device.

[0279] Step 2:

[0280] The terminal transmits the captured image data to the server using the communication infrastructure. The input is the saved image data, and the output is the completion of the transfer of the image data to the server. Through this process, the data is ready for analysis.

[0281] Step 3:

[0282] The server inputs the received image data into an analysis model using TensorFlow for image analysis. The input is the transmitted image data, and the output is the analysis result regarding the damage characteristics. Here, feature extraction is performed using deep learning technology.

[0283] Step 4:

[0284] The server compares the analysis result with the past data records in the MySQL database to determine damage. The input is the analysis result and the past data records, and the output is the determination regarding the presence or absence of damage. Through this determination, it is judged whether the item is normal or damaged.

[0285] Step 5:

[0286] Based on the determination result, the server automatically generates a report and notifies the responsible person. The input is the determination result, and the output is the generated report and its notification. The report describes the damage situation and the necessity of compensation and is made to reach the responsible person quickly.

[0287] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0288] The present invention is a system that integrates the technology of recognizing the user's emotion into the damage determination process at the time of returning the terminal. This system includes a terminal, a server, a communication network, an image analysis model, and an emotion engine.

[0289] When a user returns a device, the device's camera captures a detailed image of the device's appearance upon return. This image data is transmitted to a server via the network. Simultaneously, an emotion engine built into the device analyzes the user's facial expressions and voice, and sends the results to the server. The emotion engine has the capability to accurately determine whether the user is nervous or relieved upon returning the device.

[0290] The server analyzes the received image data using a modifiable deep learning model. The deep learning model identifies damage characteristics based on a vast amount of historical data and compares them to previous damage records. User sentiment data can also be used in the analysis, allowing the judgment method to be adjusted according to the user's state.

[0291] Based on the analysis results, the server determines the extent of damage to the returned device and generates a report. This report is presented in an appropriate format, taking into account the user's emotional state. For example, if the user is stressed when returning the device, the notification content is adjusted to be gentler and more reassuring.

[0292] As a concrete example, consider a case where a user returns a device with a small crack on the screen. If the emotion engine analyzes that the user is showing anxiety or tension regarding this situation, the server will send a notification that includes a sincere response in addition to the usual repair procedures. This will reassure the user, and the problem will be resolved.

[0293] This system not only streamlines the process of determining damage when devices are returned, but also improves the user experience.

[0294] The following describes the processing flow.

[0295] Step 1:

[0296] The user returns the terminal. When it is recognized that the use of the terminal has ended and the terminal is brought to the designated return location, the return system is started.

[0297] Step 2:

[0298] The terminal takes pictures of its appearance. The returned terminal uses the camera built into itself to take pictures of its appearance from multiple angles. At this time, the terminal automatically checks the quality of the images and takes re-pictures if necessary.

[0299] Step 3:

[0300] The terminal activates the emotion engine. The emotion engine records the user's expression and voice and analyzes the user's emotional state. For example, it determines in real time whether the user is smiling or nervous.

[0301] Step 4:

[0302] The terminal sends the images taken and the emotion data to the server. Through the communication network, the terminal uploads the analyzed user's emotion data to the server together with high-resolution image data.

[0303] Step 5:

[0304] The server analyzes the image data. The AI analysis model installed on the server uses deep learning technology to analyze the images and extracts the characteristics of the damage that occurred to the terminal.

[0305] Step 6:

[0306] The server takes the emotion data into account in the analysis. The server adjusts the damage assessment by referring to the user's emotion data and makes a more appropriate judgment.

[0307] Step 7:

[0308] The server performs damage assessment and generates a report. Based on the analysis results, it determines whether the terminal's condition is due to intentional damage or normal wear and tear, and generates a detailed report.

[0309] Step 8:

[0310] The server notifies the user. Considering the report's contents, it creates an appropriate notification message tailored to the user's emotional state and sends it to the user via email, app notifications, or other means.

[0311] (Example 2)

[0312] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0313] Conventional equipment return damage assessment processes merely assessed and reported only the physical damage to the equipment, lacking consideration for the user's psychological state. As a result, users were not provided with adequate reassurance or support, sometimes leading to a poor user experience. This invention aims to improve the user experience by considering the user's emotions during the return process and providing more appropriate assessment and reporting.

[0314] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0315] In this invention, the server includes means for acquiring return information after use and user emotional information, means for analyzing the accumulated image data and emotional information, and means for reporting the damage status of the equipment based on the analysis results and notifying the user in a format that takes into account their emotional state. This enables comprehensive judgment and reporting that takes into account not only physical damage but also the user's psychological state.

[0316] "Return information" refers to data acquired when a user returns equipment, and it indicates the status of the return and its physical condition.

[0317] "Emotional information" refers to data that indicates the user's psychological state, extracted from their facial expressions and voice, and is used to identify the user's emotions at the time of return.

[0318] A "communication network" is a system that exchanges data via various wired or wireless technologies, using paths and media for transmitting information.

[0319] An "analysis algorithm" is a procedure or method for processing large amounts of data and extracting or determining information according to a specific purpose.

[0320] A "machine learning algorithm" is a set of methods that automatically learn from past data and experience to perform identification and prediction, and is used to recognize patterns in data.

[0321] "Damage assessment" refers to the process of identifying and evaluating whether there is any physical or functional damage to equipment, and to what extent.

[0322] "Notification" refers to the act of conveying a decision or information to a user, and involves sending an appropriate message depending on the situation.

[0323] This system is an integrated system designed to take into account the user's emotions when equipment is returned, enabling more appropriate damage assessment and reporting. It mainly consists of a combination of terminals, servers, communication networks, data analysis algorithms, and an emotion analysis engine.

[0324] The device automatically activates its camera when the user returns the equipment, taking a picture of the equipment's appearance upon return. In addition, it records the user's emotional information using facial recognition and voice analysis technologies. This includes common computer vision techniques for face detection and software for voice analysis. This information is transmitted to a server using wireless communication technologies (e.g., Wi-Fi or mobile networks).

[0325] The server processes the received image data and sentiment data. The image data is analyzed using machine learning algorithms to identify corruption features. It is preferable to use deep learning frameworks such as TensorFlow or PyTorch for the analysis algorithms. The analysis results are compared with historical databases, and if new corruption is detected, its details are reflected in the report.

[0326] Emotional data is used to determine the user's psychological state. The emotion analysis engine analyzes the emotions the user is experiencing at the time of return based on collected facial and voice data. The resulting emotional state is used to adjust the content and tone of the report. For example, if the user is feeling anxious, the report will be written in a more considerate manner, and additional reassuring information will be provided.

[0327] As a concrete example, consider a case where a user returns a smartphone with a small crack in the screen. If the emotion analysis engine determines that the user appears anxious, the server will generate a reassuring notification in addition to the usual damage assessment. This allows the user to feel that they are receiving a sincere response and to have confidence that the problem will be resolved.

[0328] Example of a prompt:

[0329] "Please explain how you will analyze the emotions users experience when returning their devices and how you will adjust the damage assessment process based on those results."

[0330] This system enables efficient and highly satisfying service delivery through a comprehensive approach that considers not only physical damage but also the user's psychological state.

[0331] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0332] Step 1:

[0333] The device automatically activates its built-in camera upon return and takes a picture of the device's exterior. The captured image data is then processed, including exposure correction and noise reduction, before being sent to the server. The input is the physical exterior of the device, and the output is the processed image data.

[0334] Step 2:

[0335] The device records the user's facial expressions and voice, and analyzes them in real time using an emotion analysis engine. This analysis utilizes facial recognition technology and voice feature analysis to identify the user's emotional state (e.g., tension, relief, anxiety). Input consists of the user's facial image and voice data, and output is the determined emotion information.

[0336] Step 3:

[0337] The server feeds the received image data into a machine learning algorithm to extract damage features. This process uses a conventional deep learning model and compares the data against a historical database. The input is image data, and the output is information indicating the location and characteristics of the damage.

[0338] Step 4:

[0339] The server analyzes emotional information and evaluates the user's psychological state. Based on this information, it adjusts the criteria for damage assessment to perform a more accurate evaluation. The input is emotional information, and the output is the adjusted damage assessment criteria.

[0340] Step 5:

[0341] The server generates a report on the equipment's damage status based on the integrated analysis results. Since the content and tone of this report are adjusted according to the user's emotional state, the inputs are damage information and emotional information, and the output is a customized report for the user.

[0342] (Application Example 2)

[0343] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0344] There is a need for a system that simultaneously improves damage assessment and user experience when returning consumer robots. In particular, the challenge is to enable flexible responses that take into account the user's emotions and reduce stress during the return process.

[0345] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0346] In this invention, the server includes means for acquiring return information after use, means for photographing the appearance of the equipment, means for accumulating the captured image data via a communication network, means for analyzing the accumulated image data using an analysis model, means for analyzing the user's emotions, and means for adaptively adjusting the report content based on the results of the emotion analysis means. This not only enables efficient determination of equipment damage upon return but also allows for flexible responses in accordance with the user's psychological state.

[0347] "Means for obtaining return information after use" refers to a device or method that has the function of obtaining information regarding the return of equipment when its use has ended.

[0348] "Means for photographing the appearance of equipment" refers to a device or method for photographing the appearance of equipment as an image in order to record the surface condition and characteristics of the equipment.

[0349] "Means of aggregating information via a communication network" refers to a device or method that has the function of collecting information through a network into a centralized database or server.

[0350] An "analysis model" is a computational model built based on past data and learning results in order to analyze acquired data.

[0351] "Emotional analysis means" refers to a device or method for detecting and analyzing emotions from a user's facial expressions, voice, etc.

[0352] "Means of adaptive adjustment" refers to a device or method that has the function of automatically changing the optimal response according to the situation based on the analysis results.

[0353] The system for carrying out this invention mainly includes a server, a terminal, a communication network, a camera, and an emotion analysis engine. The terminal is equipped with a camera that acquires return information after use and also takes pictures of the device's appearance. The captured image data is transmitted to the server via the communication network.

[0354] After receiving this image data, the server analyzes the data using an analysis model. This analysis model is built on a deep learning algorithm and determines equipment damage by comparing past data records with current image data.

[0355] Furthermore, the emotion analysis engine installed in the device analyzes the user's emotions from their facial expressions and voice. This emotional data is sent to a server and reflected in the damage assessment report. Based on this information, the server adaptively adjusts the report content according to the user's emotions and notifies the user. This notification is designed to be flexible and considerate using a generative AI model.

[0356] For example, if a user returns equipment and notices minor cosmetic damage, and the user expresses tension or anxiety, the system will generate a notification that uses gentle language to explain the situation and provide reassurance.

[0357] An example of a prompt message is, "Generate thoughtful feedback based on the user's feelings and the device's condition upon return."

[0358] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0359] Step 1:

[0360] The terminal retrieves return information and uses its camera to photograph the exterior of the returned equipment. The input is the returned equipment, and the output is an image of the equipment's exterior. The terminal uses its camera to take detailed photographs of the entire front surface of the equipment.

[0361] Step 2:

[0362] The device transmits image data it captures to a server via a communication network. The input is an image of the device's exterior, and the output is the image data transmitted to the server. The device performs appropriate compression and encryption to securely upload the data to the server.

[0363] Step 3:

[0364] The server inputs the received image data into an analysis model and analyzes the data. The input is the image data sent to the server, and the output is a judgment result indicating the presence and extent of damage. The server utilizes a deep learning algorithm to identify damage characteristics by comparing them with past data.

[0365] Step 4:

[0366] The device detects the user's facial expressions and voice, and analyzes those emotions using an emotion analysis engine. The input is the user's facial image and voice data, and the output is data indicating the user's emotional state. The device performs emotion analysis in real time, determining things like tension and a sense of security.

[0367] Step 5:

[0368] The server receives emotion analysis data and generates a damage assessment report based on it. The input is the damage assessment result and the user's emotion data, and the output is the report content to be notified to the user. The server uses a generation AI model and utilizes prompt text to generate feedback content.

[0369] Step 6:

[0370] The user receives the report and chooses further actions based on it. The input is the report information notified to the user, and the output is the user's next action or feedback. The user can proceed with the return process with confidence.

[0371] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0372] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0373] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0374] [Third Embodiment]

[0375] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0376] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0377] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0378] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0379] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0380] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0381] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0382] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0383] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0384] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0385] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0386] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0387] The present invention provides a system that automatically determines the extent of damage to a terminal during the terminal return process. The system includes a server, terminals, a communication network, and an AI-based analysis model.

[0388] First, the user returns the device after use. Once the device is returned, the built-in camera takes pictures of the device's exterior from multiple angles. These images are acquired in high resolution and temporarily stored in the device's storage. The device then sends the captured images to the server via the communication network.

[0389] The server inputs the received image data into an AI analysis model. This analysis model uses deep learning technology and analyzes the images based on various damage patterns it has learned in the past. During feature extraction, it is designed to identify damage patterns such as scratches, cracks, and discoloration. The analysis results are compared with past damage cases recorded in the server's database.

[0390] Once the server finishes its assessment, a result is generated. Based on this result, it determines whether the returned device was intentionally damaged or whether the damage was due to normal use. Through this automated assessment, the server generates a report and notifies the user.

[0391] As a concrete example, consider a case where a cracked screen is found on a user's returned device. The server analyzes the image taken by the device, and the AI ​​model recognizes it as intentional damage. If it is determined to be new damage through comparison with past data, the server will determine that compensation is required and record this in the report. The user will then be notified and provided with information about the repair costs.

[0392] This system automates the previously reliant manual damage assessment process, enabling faster and more accurate determinations.

[0393] The following describes the processing flow.

[0394] Step 1:

[0395] The user returns the device. The system's return process begins when the user returns the device to the designated location after use.

[0396] Step 2:

[0397] The device takes pictures of its exterior. Upon receiving a returned device, it uses its built-in camera to take high-resolution images of its exterior from multiple angles. The captured images are temporarily stored on the device.

[0398] Step 3:

[0399] The device sends the image to the server. The captured image data is compressed via the communication network and securely uploaded to the server.

[0400] Step 4:

[0401] The server analyzes the images. The server inputs the received images into an AI analysis model, which uses deep learning techniques to extract damage characteristics. This model is trained to identify patterns of damage and failure.

[0402] Step 5:

[0403] The server compares the current situation to past database data. Based on the features extracted by the AI ​​model, the server compares the current situation to past damage cases in the database. It calculates similarity and determines whether the damage is existing wear and tear or new damage.

[0404] Step 6:

[0405] The server determines the result and generates a report. Based on the determination, the server generates a report detailing the necessary repayments if any are required.

[0406] Step 7:

[0407] The server notifies the user. Based on the report, the user is notified of the result of the device return and, if necessary, is provided with information regarding compensation claims.

[0408] (Example 1)

[0409] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0410] In recent years, as the use of information terminals has expanded, there has been a demand for faster and fairer damage assessments upon terminal return. However, manual assessments are subjective and can lack fairness. Furthermore, while rapid response is required, efficient methods for assessing terminal condition have not yet been sufficiently established. Therefore, there is a need for automated technology that can fairly and quickly assess terminal damage and take appropriate measures.

[0411] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0412] In this invention, the server includes means for acquiring information regarding the return of an information terminal, means for capturing images of the information terminal's appearance from multiple viewpoints, means for accumulating image data via an information communication network, means for analyzing data using an analysis model, and means for notifying the user of the determination results. This makes it possible to automatically and quickly evaluate the damage status of the terminal and take fair and appropriate measures for the user.

[0413] "Information regarding the return of an information terminal after use" refers to a series of data and information recorded when a user returns an information terminal after using it.

[0414] An "information terminal" is an electronic device that a user can operate and use, and includes devices such as computers, smartphones, and tablets.

[0415] "Imaging means" refers to a device or method used to photograph the external appearance of an information terminal, and utilizes a device such as a camera.

[0416] An "information and communication network" refers to communication infrastructure such as the internet and mobile networks used to send and receive data.

[0417] An "analytical model" is an algorithm or software used to analyze given data, and includes techniques such as machine learning and deep learning.

[0418] The "judgment result" is the result of the analysis obtained by the analytical model, and indicates a judgment regarding the damage status of the information terminal.

[0419] A "claim for damages" is the act of claiming necessary expenses or compensation based on whether or not the returned information terminal was damaged and to what extent.

[0420] The system of the present invention is for automating the assessment of damage to information terminals upon their return. Specifically, it comprises an information terminal, a server, a communication network, and a generative AI model.

[0421] After use, the user places the information terminal in a designated return location. The terminal automatically uses its built-in camera to capture images of its exterior from multiple angles. High-resolution image data is temporarily stored on the terminal. The stored image data is then transmitted to a server via a communication network. Wi-Fi or a mobile network is used during this process, and the transmitted image data is encrypted.

[0422] The server inputs the received image data into an analysis model that uses machine learning techniques. This analysis model is based on common deep learning frameworks such as TensorFlow and PyTorch, and analyzes the images using a variety of damage patterns that it has learned in the past. It can automatically extract characteristic damage patterns, such as scratches, cracks, and discoloration.

[0423] The analysis results are compared with historical information stored in the server's database to determine whether the damage was intentional or due to normal use. A report is then automatically generated and sent to the user via email. The report details the damage and necessary actions (e.g., repair costs).

[0424] Example prompt: "Analyze the images of the returned device to determine if there is any damage and what the cause is."

[0425] This system enables a rapid and fair assessment of the device's damage status and allows for appropriate countermeasures to be taken.

[0426] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0427] Step 1:

[0428] The user returns the information terminal to a designated location. The terminal automatically activates its camera and takes pictures of its exterior from various angles. The input is external information of the terminal, and the output is high-resolution image data. This operation collects data to understand the terminal's status from all angles.

[0429] Step 2:

[0430] The device temporarily stores captured image data in its internal storage. Simultaneously, once a network connection is established, it transmits the image data to the server via the communication network. The input is the data stored on the device, and the output is the received data on the server. Data encryption protocols are used during communication to ensure security.

[0431] Step 3:

[0432] The server inputs the received image data into a machine learning model and begins the analysis. The model uses a framework such as TensorFlow to extract features based on previously trained datasets. The input is image data, and the output is detection information regarding damage such as scratches, cracks, and discoloration. This process automatically evaluates the device's damage status.

[0433] Step 4:

[0434] The server identifies the cause of the damage by comparing the analysis results with past cases in its internal database. The inputs are the analysis results and past case data, while the output is damage assessment and judgment information. By identifying similar cases and patterns, it determines whether the damage was intentional.

[0435] Step 5:

[0436] The server generates a report based on the assessment information and notifies the user via email. The report includes details of the damage, the reason for the assessment, and the necessary procedures. The input is the damage assessment information, and the output is the generated report. By quickly delivering this information to the user, the server enables appropriate action.

[0437] (Application Example 1)

[0438] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0439] In logistics centers and other inventory management facilities, the inspection of goods during package receipt and shipment relies on manual processes, making judgment criteria subjective and hindering quick and accurate damage assessment. Furthermore, there is a need to automate the process for making fair judgments regarding damage and the need for compensation.

[0440] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0441] In this invention, the server includes means for acquiring return information after use, means for photographing the exterior based on the return information, means for accumulating the captured image data via a communication infrastructure, means for analyzing the accumulated image data using an analysis model, means for comparing the results of the analysis model with past data records to determine damage, means for reporting the condition of the returned equipment based on the determination result, and means for determining compensation based on the damage result and creating a report for the person in charge. This enables rapid and objective determination of package damage and automation of fair compensation decisions.

[0442] "Return information" refers to the data associated with a used item or device being returned to the administrator, including recorded history and status.

[0443] "Exterior" refers to the parts that make up the outside of an item or piece of equipment, and describes its appearance and physical condition.

[0444] A "communication infrastructure" is a network infrastructure used to send and receive digital data, and is a system that can transmit information quickly.

[0445] An "analytical model" is a mathematical or algorithmic structure used to analyze collected data and extract meaningful information.

[0446] "Damage assessment" is the process of determining whether an item or piece of equipment is in good condition or damaged.

[0447] A "report" is a written or digital document that summarizes analysis and judgment results and systematically describes related information.

[0448] The system for realizing this invention will be used in inventory management facilities such as logistics centers. When used items are returned to the facility, a terminal is used to photograph the exterior. The terminal is a smartphone or tablet equipped with a high-resolution camera, and it photographs the exterior of the item from multiple angles. The captured image data is temporarily stored in the terminal and then transmitted to a server using a communication infrastructure.

[0449] The server processes the received image data and analyzes it using an analysis model based on machine learning algorithms. This analysis utilizes deep learning frameworks such as TensorFlow and Python to extract damage features. The analysis results are compared with historical data records to determine the extent of the damage. Based on the determination, a report is automatically generated and notified to the responsible party. The generated report is properly stored and managed in a MySQL database.

[0450] As a concrete example, when a package of confectionery arrives at the distribution center, the terminal takes a picture of the package, and the data is analyzed on a server. The AI ​​model identifies a new dent in the box, and the report states, "Damage found. Compensation required." The results are quickly notified to the person in charge through the application.

[0451] An example of a prompt statement to be input into the generating AI model is, "Analyze images of packages in transit at the logistics center and identify any damage." This prompt statement helps to make more accurate damage detections.

[0452] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0453] Step 1:

[0454] The device takes high-resolution photographs of the exterior of used items from multiple angles. The input is the actual item, and the output is the captured image data. The captured image data is temporarily stored within the device.

[0455] Step 2:

[0456] The device transmits the captured image data to the server using the communication infrastructure. The input is the stored image data, and the output is the completion of the transfer of the image data to the server. This process prepares the data for analysis.

[0457] Step 3:

[0458] The server inputs the received image data into an analysis model using TensorFlow and performs image analysis. The input is the transmitted image data, and the output is the analysis results regarding the damage features. Here, feature extraction is performed using deep learning techniques.

[0459] Step 4:

[0460] The server compares the analysis results with historical data records and a MySQL database to determine if there is damage. The input consists of the analysis results and historical data records, and the output is a determination of whether or not there is damage. This determination determines whether the item is normal or damaged.

[0461] Step 5:

[0462] The server automatically generates a report based on the assessment result and notifies the responsible person. The input is the assessment result, and the output is the generated report and the notification. The report includes details of the damage and whether compensation is required, and is designed to reach the responsible person quickly.

[0463] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0464] This invention relates to a system that integrates user emotion recognition technology into the damage assessment process when a device is returned. The system includes a device, a server, a communication network, an image analysis model, and an emotion engine.

[0465] When a user returns a device, the device's camera captures a detailed image of the device's appearance upon return. This image data is transmitted to a server via the network. Simultaneously, an emotion engine built into the device analyzes the user's facial expressions and voice, and sends the results to the server. The emotion engine has the capability to accurately determine whether the user is nervous or relieved upon returning the device.

[0466] The server analyzes the received image data using a modifiable deep learning model. The deep learning model identifies damage characteristics based on a vast amount of historical data and compares them to previous damage records. User sentiment data can also be used in the analysis, allowing the judgment method to be adjusted according to the user's state.

[0467] Based on the analysis results, the server determines the extent of damage to the returned device and generates a report. This report is presented in an appropriate format, taking into account the user's emotional state. For example, if the user is stressed when returning the device, the notification content is adjusted to be gentler and more reassuring.

[0468] As a concrete example, consider a case where a user returns a device with a small crack on the screen. If the emotion engine analyzes that the user is showing anxiety or tension regarding this situation, the server will send a notification that includes a sincere response in addition to the usual repair procedures. This will reassure the user, and the problem will be resolved.

[0469] This system not only streamlines the process of determining damage when devices are returned, but also improves the user experience.

[0470] The following describes the processing flow.

[0471] Step 1:

[0472] The user returns the device. The return system is activated when the user acknowledges that they have finished using the device and brings it to the designated return location.

[0473] Step 2:

[0474] The device takes pictures of its exterior. Upon receiving the device, it uses its built-in camera to take pictures of its exterior from multiple angles. During this process, the device automatically checks the image quality and retakes the pictures if necessary.

[0475] Step 3:

[0476] The device activates the emotion engine. The emotion engine records the user's facial expressions and voice, and analyzes the user's emotional state. For example, it can determine in real time whether the user is smiling or feeling nervous.

[0477] Step 4:

[0478] The device sends images and emotion data captured by the device to the server. Through the communication network, the device uploads high-resolution image data along with the analyzed user emotion data to the server.

[0479] Step 5:

[0480] The server analyzes the image data. An AI analysis model installed on the server uses deep learning technology to analyze the images and extract the characteristics of the damage that occurred on the device.

[0481] Step 6:

[0482] The server incorporates emotional data into its analysis. The server uses the user's emotional data to adjust the damage assessment and make more appropriate judgments.

[0483] Step 7:

[0484] The server performs damage assessment and generates a report. Based on the analysis results, it determines whether the terminal's condition is due to intentional damage or normal wear and tear, and generates a detailed report.

[0485] Step 8:

[0486] The server notifies the user. Considering the report's contents, it creates an appropriate notification message tailored to the user's emotional state and sends it to the user via email, app notifications, or other means.

[0487] (Example 2)

[0488] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] Conventional equipment return damage assessment processes merely assessed and reported only the physical damage to the equipment, lacking consideration for the user's psychological state. As a result, users were not provided with adequate reassurance or support, sometimes leading to a poor user experience. This invention aims to improve the user experience by considering the user's emotions during the return process and providing more appropriate assessment and reporting.

[0490] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0491] In this invention, the server includes means for acquiring return information after use and user emotional information, means for analyzing the accumulated image data and emotional information, and means for reporting the damage status of the equipment based on the analysis results and notifying the user in a format that takes into account their emotional state. This enables comprehensive judgment and reporting that takes into account not only physical damage but also the user's psychological state.

[0492] "Return information" refers to data acquired when a user returns equipment, and it indicates the status of the return and its physical condition.

[0493] "Emotional information" refers to data that indicates the user's psychological state, extracted from their facial expressions and voice, and is used to identify the user's emotions at the time of return.

[0494] A "communication network" is a system that exchanges data via various wired or wireless technologies, using paths and media for transmitting information.

[0495] An "analysis algorithm" is a procedure or method for processing large amounts of data and extracting or determining information according to a specific purpose.

[0496] A "machine learning algorithm" is a set of methods that automatically learn from past data and experience to perform identification and prediction, and is used to recognize patterns in data.

[0497] "Damage assessment" refers to the process of identifying and evaluating whether there is any physical or functional damage to equipment, and to what extent.

[0498] "Notification" refers to the act of conveying a decision or information to a user, and involves sending an appropriate message depending on the situation.

[0499] This system is an integrated system designed to take into account the user's emotions when equipment is returned, enabling more appropriate damage assessment and reporting. It mainly consists of a combination of terminals, servers, communication networks, data analysis algorithms, and an emotion analysis engine.

[0500] The device automatically activates its camera when the user returns the equipment, taking a picture of the equipment's appearance upon return. In addition, it records the user's emotional information using facial recognition and voice analysis technologies. This includes common computer vision techniques for face detection and software for voice analysis. This information is transmitted to a server using wireless communication technologies (e.g., Wi-Fi or mobile networks).

[0501] The server processes the received image data and sentiment data. The image data is analyzed using machine learning algorithms to identify corruption features. It is preferable to use deep learning frameworks such as TensorFlow or PyTorch for the analysis algorithms. The analysis results are compared with historical databases, and if new corruption is detected, its details are reflected in the report.

[0502] Emotional data is used to determine the user's psychological state. The emotion analysis engine analyzes the emotions the user is experiencing at the time of return based on collected facial and voice data. The resulting emotional state is used to adjust the content and tone of the report. For example, if the user is feeling anxious, the report will be written in a more considerate manner, and additional reassuring information will be provided.

[0503] As a concrete example, consider a case where a user returns a smartphone with a small crack in the screen. If the emotion analysis engine determines that the user appears anxious, the server will generate a reassuring notification in addition to the usual damage assessment. This allows the user to feel that they are receiving a sincere response and to have confidence that the problem will be resolved.

[0504] Example of a prompt:

[0505] "Please explain how you will analyze the emotions users experience when returning their devices and how you will adjust the damage assessment process based on those results."

[0506] This system enables efficient and highly satisfying service delivery through a comprehensive approach that considers not only physical damage but also the user's psychological state.

[0507] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0508] Step 1:

[0509] The device automatically activates its built-in camera upon return and takes a picture of the device's exterior. The captured image data is then processed, including exposure correction and noise reduction, before being sent to the server. The input is the physical exterior of the device, and the output is the processed image data.

[0510] Step 2:

[0511] The device records the user's facial expressions and voice, and analyzes them in real time using an emotion analysis engine. This analysis utilizes facial recognition technology and voice feature analysis to identify the user's emotional state (e.g., tension, relief, anxiety). Input consists of the user's facial image and voice data, and output is the determined emotion information.

[0512] Step 3:

[0513] The server feeds the received image data into a machine learning algorithm to extract damage features. This process uses a conventional deep learning model and compares the data against a historical database. The input is image data, and the output is information indicating the location and characteristics of the damage.

[0514] Step 4:

[0515] The server analyzes emotional information and evaluates the user's psychological state. Based on this information, it adjusts the criteria for damage assessment to perform a more accurate evaluation. The input is emotional information, and the output is the adjusted damage assessment criteria.

[0516] Step 5:

[0517] The server generates a report on the equipment's damage status based on the integrated analysis results. Since the content and tone of this report are adjusted according to the user's emotional state, the inputs are damage information and emotional information, and the output is a customized report for the user.

[0518] (Application Example 2)

[0519] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] There is a need for a system that simultaneously improves damage assessment and user experience when returning consumer robots. In particular, the challenge is to enable flexible responses that take into account the user's emotions and reduce stress during the return process.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0522] In this invention, the server includes means for acquiring return information after use, means for photographing the appearance of the equipment, means for accumulating the captured image data via a communication network, means for analyzing the accumulated image data using an analysis model, means for analyzing the user's emotions, and means for adaptively adjusting the report content based on the results of the emotion analysis means. This not only enables efficient determination of equipment damage upon return but also allows for flexible responses in accordance with the user's psychological state.

[0523] "Means for obtaining return information after use" refers to a device or method that has the function of obtaining information regarding the return of equipment when its use has ended.

[0524] "Means for photographing the appearance of equipment" refers to a device or method for photographing the appearance of equipment as an image in order to record the surface condition and characteristics of the equipment.

[0525] "Means of aggregating information via a communication network" refers to a device or method that has the function of collecting information through a network into a centralized database or server.

[0526] An "analysis model" is a computational model built based on past data and learning results in order to analyze acquired data.

[0527] "Emotional analysis means" refers to a device or method for detecting and analyzing emotions from a user's facial expressions, voice, etc.

[0528] "Means of adaptive adjustment" refers to a device or method that has the function of automatically changing the optimal response according to the situation based on the analysis results.

[0529] The system for carrying out this invention mainly includes a server, a terminal, a communication network, a camera, and an emotion analysis engine. The terminal is equipped with a camera that acquires return information after use and also takes pictures of the device's appearance. The captured image data is transmitted to the server via the communication network.

[0530] After receiving this image data, the server analyzes the data using an analysis model. This analysis model is built on a deep learning algorithm and determines equipment damage by comparing past data records with current image data.

[0531] Furthermore, the emotion analysis engine installed in the device analyzes the user's emotions from their facial expressions and voice. This emotional data is sent to a server and reflected in the damage assessment report. Based on this information, the server adaptively adjusts the report content according to the user's emotions and notifies the user. This notification is designed to be flexible and considerate using a generative AI model.

[0532] For example, if a user returns equipment and notices minor cosmetic damage, and the user expresses tension or anxiety, the system will generate a notification that uses gentle language to explain the situation and provide reassurance.

[0533] An example of a prompt message is, "Generate thoughtful feedback based on the user's feelings and the device's condition upon return."

[0534] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0535] Step 1:

[0536] The terminal retrieves return information and uses its camera to photograph the exterior of the returned equipment. The input is the returned equipment, and the output is an image of the equipment's exterior. The terminal uses its camera to take detailed photographs of the entire front surface of the equipment.

[0537] Step 2:

[0538] The device transmits image data it captures to a server via a communication network. The input is an image of the device's exterior, and the output is the image data transmitted to the server. The device performs appropriate compression and encryption to securely upload the data to the server.

[0539] Step 3:

[0540] The server inputs the received image data into an analysis model and analyzes the data. The input is the image data sent to the server, and the output is a judgment result indicating the presence and extent of damage. The server utilizes a deep learning algorithm to identify damage characteristics by comparing them with past data.

[0541] Step 4:

[0542] The device detects the user's facial expressions and voice, and analyzes those emotions using an emotion analysis engine. The input is the user's facial image and voice data, and the output is data indicating the user's emotional state. The device performs emotion analysis in real time, determining things like tension and a sense of security.

[0543] Step 5:

[0544] The server receives emotion analysis data and generates a damage assessment report based on it. The input is the damage assessment result and the user's emotion data, and the output is the report content to be notified to the user. The server uses a generation AI model and utilizes prompt text to generate feedback content.

[0545] Step 6:

[0546] The user receives the report and chooses further actions based on it. The input is the report information notified to the user, and the output is the user's next action or feedback. The user can proceed with the return process with confidence.

[0547] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0548] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0549] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0550] [Fourth Embodiment]

[0551] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0552] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0553] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0554] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0555] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0556] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0557] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0558] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0559] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0560] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0561] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0562] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0563] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0564] The present invention provides a system that automatically determines the extent of damage to a terminal during the terminal return process. The system includes a server, terminals, a communication network, and an AI-based analysis model.

[0565] First, the user returns the device after use. Once the device is returned, the built-in camera takes pictures of the device's exterior from multiple angles. These images are acquired in high resolution and temporarily stored in the device's storage. The device then sends the captured images to the server via the communication network.

[0566] The server inputs the received image data into an AI analysis model. This analysis model uses deep learning technology and analyzes the images based on various damage patterns it has learned in the past. During feature extraction, it is designed to identify damage patterns such as scratches, cracks, and discoloration. The analysis results are compared with past damage cases recorded in the server's database.

[0567] Once the server finishes its assessment, a result is generated. Based on this result, it determines whether the returned device was intentionally damaged or whether the damage was due to normal use. Through this automated assessment, the server generates a report and notifies the user.

[0568] As a concrete example, consider a case where a cracked screen is found on a user's returned device. The server analyzes the image taken by the device, and the AI ​​model recognizes it as intentional damage. If it is determined to be new damage through comparison with past data, the server will determine that compensation is required and record this in the report. The user will then be notified and provided with information about the repair costs.

[0569] This system automates the previously reliant manual damage assessment process, enabling faster and more accurate determinations.

[0570] The following describes the processing flow.

[0571] Step 1:

[0572] The user returns the device. The system's return process begins when the user returns the device to the designated location after use.

[0573] Step 2:

[0574] The device takes pictures of its exterior. Upon receiving a returned device, it uses its built-in camera to take high-resolution images of its exterior from multiple angles. The captured images are temporarily stored on the device.

[0575] Step 3:

[0576] The device sends the image to the server. The captured image data is compressed via the communication network and securely uploaded to the server.

[0577] Step 4:

[0578] The server analyzes the images. The server inputs the received images into an AI analysis model, which uses deep learning techniques to extract damage characteristics. This model is trained to identify patterns of damage and failure.

[0579] Step 5:

[0580] The server compares the current situation to past database data. Based on the features extracted by the AI ​​model, the server compares the current situation to past damage cases in the database. It calculates similarity and determines whether the damage is existing wear and tear or new damage.

[0581] Step 6:

[0582] The server determines the result and generates a report. Based on the determination, the server generates a report detailing the necessary repayments if any are required.

[0583] Step 7:

[0584] The server notifies the user. Based on the report, the user is notified of the result of the device return and, if necessary, is provided with information regarding compensation claims.

[0585] (Example 1)

[0586] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0587] In recent years, as the use of information terminals has expanded, there has been a demand for faster and fairer damage assessments upon terminal return. However, manual assessments are subjective and can lack fairness. Furthermore, while rapid response is required, efficient methods for assessing terminal condition have not yet been sufficiently established. Therefore, there is a need for automated technology that can fairly and quickly assess terminal damage and take appropriate measures.

[0588] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0589] In this invention, the server includes means for acquiring information regarding the return of an information terminal, means for capturing images of the information terminal's appearance from multiple viewpoints, means for accumulating image data via an information communication network, means for analyzing data using an analysis model, and means for notifying the user of the determination results. This makes it possible to automatically and quickly evaluate the damage status of the terminal and take fair and appropriate measures for the user.

[0590] "Information regarding the return of an information terminal after use" refers to a series of data and information recorded when a user returns an information terminal after using it.

[0591] An "information terminal" is an electronic device that a user can operate and use, and includes devices such as computers, smartphones, and tablets.

[0592] "Imaging means" refers to a device or method used to photograph the external appearance of an information terminal, and utilizes a device such as a camera.

[0593] An "information and communication network" refers to communication infrastructure such as the internet and mobile networks used to send and receive data.

[0594] An "analytical model" is an algorithm or software used to analyze given data, and includes techniques such as machine learning and deep learning.

[0595] The "judgment result" is the result of the analysis obtained by the analytical model, and indicates a judgment regarding the damage status of the information terminal.

[0596] A "claim for damages" is the act of claiming necessary expenses or compensation based on whether or not the returned information terminal was damaged and to what extent.

[0597] The system of the present invention is for automating the assessment of damage to information terminals upon their return. Specifically, it comprises an information terminal, a server, a communication network, and a generative AI model.

[0598] After use, the user places the information terminal in a designated return location. The terminal automatically uses its built-in camera to capture images of its exterior from multiple angles. High-resolution image data is temporarily stored on the terminal. The stored image data is then transmitted to a server via a communication network. Wi-Fi or a mobile network is used during this process, and the transmitted image data is encrypted.

[0599] The server inputs the received image data into an analysis model that uses machine learning techniques. This analysis model is based on common deep learning frameworks such as TensorFlow and PyTorch, and analyzes the images using a variety of damage patterns that it has learned in the past. It can automatically extract characteristic damage patterns, such as scratches, cracks, and discoloration.

[0600] The analysis results are compared with historical information stored in the server's database to determine whether the damage was intentional or due to normal use. A report is then automatically generated and sent to the user via email. The report details the damage and necessary actions (e.g., repair costs).

[0601] Example prompt: "Analyze the images of the returned device to determine if there is any damage and what the cause is."

[0602] This system enables a rapid and fair assessment of the device's damage status and allows for appropriate countermeasures to be taken.

[0603] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0604] Step 1:

[0605] The user returns the information terminal to a designated location. The terminal automatically activates its camera and takes pictures of its exterior from various angles. The input is external information of the terminal, and the output is high-resolution image data. This operation collects data to understand the terminal's status from all angles.

[0606] Step 2:

[0607] The device temporarily stores captured image data in its internal storage. Simultaneously, once a network connection is established, it transmits the image data to the server via the communication network. The input is the data stored on the device, and the output is the received data on the server. Data encryption protocols are used during communication to ensure security.

[0608] Step 3:

[0609] The server inputs the received image data into a machine learning model and begins the analysis. The model uses a framework such as TensorFlow to extract features based on previously trained datasets. The input is image data, and the output is detection information regarding damage such as scratches, cracks, and discoloration. This process automatically evaluates the device's damage status.

[0610] Step 4:

[0611] The server identifies the cause of the damage by comparing the analysis results with past cases in its internal database. The inputs are the analysis results and past case data, while the output is damage assessment and judgment information. By identifying similar cases and patterns, it determines whether the damage was intentional.

[0612] Step 5:

[0613] The server generates a report based on the assessment information and notifies the user via email. The report includes details of the damage, the reason for the assessment, and the necessary procedures. The input is the damage assessment information, and the output is the generated report. By quickly delivering this information to the user, the server enables appropriate action.

[0614] (Application Example 1)

[0615] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] In logistics centers and other inventory management facilities, the inspection of goods during package receipt and shipment relies on manual processes, making judgment criteria subjective and hindering quick and accurate damage assessment. Furthermore, there is a need to automate the process for making fair judgments regarding damage and the need for compensation.

[0617] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0618] In this invention, the server includes means for acquiring return information after use, means for photographing the exterior based on the return information, means for accumulating the captured image data via a communication infrastructure, means for analyzing the accumulated image data using an analysis model, means for comparing the results of the analysis model with past data records to determine damage, means for reporting the condition of the returned equipment based on the determination result, and means for determining compensation based on the damage result and creating a report for the person in charge. This enables rapid and objective determination of package damage and automation of fair compensation decisions.

[0619] "Return information" refers to the data associated with a used item or device being returned to the administrator, including recorded history and status.

[0620] "Exterior" refers to the parts that make up the outside of an item or piece of equipment, and describes its appearance and physical condition.

[0621] A "communication infrastructure" is a network infrastructure used to send and receive digital data, and is a system that can transmit information quickly.

[0622] An "analytical model" is a mathematical or algorithmic structure used to analyze collected data and extract meaningful information.

[0623] "Damage assessment" is the process of determining whether an item or piece of equipment is in good condition or damaged.

[0624] A "report" is a written or digital document that summarizes analysis and judgment results and systematically describes related information.

[0625] The system for realizing this invention will be used in inventory management facilities such as logistics centers. When used items are returned to the facility, a terminal is used to photograph the exterior. The terminal is a smartphone or tablet equipped with a high-resolution camera, and it photographs the exterior of the item from multiple angles. The captured image data is temporarily stored in the terminal and then transmitted to a server using a communication infrastructure.

[0626] The server processes the received image data and analyzes it using an analysis model based on machine learning algorithms. This analysis utilizes deep learning frameworks such as TensorFlow and Python to extract damage features. The analysis results are compared with historical data records to determine the extent of the damage. Based on the determination, a report is automatically generated and notified to the responsible party. The generated report is properly stored and managed in a MySQL database.

[0627] As a concrete example, when a package of confectionery arrives at the distribution center, the terminal takes a picture of the package, and the data is analyzed on a server. The AI ​​model identifies a new dent in the box, and the report states, "Damage found. Compensation required." The results are quickly notified to the person in charge through the application.

[0628] An example of a prompt statement to be input into the generating AI model is, "Analyze images of packages in transit at the logistics center and identify any damage." This prompt statement helps to make more accurate damage detections.

[0629] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0630] Step 1:

[0631] The device takes high-resolution photographs of the exterior of used items from multiple angles. The input is the actual item, and the output is the captured image data. The captured image data is temporarily stored within the device.

[0632] Step 2:

[0633] The device transmits the captured image data to the server using the communication infrastructure. The input is the stored image data, and the output is the completion of the transfer of the image data to the server. This process prepares the data for analysis.

[0634] Step 3:

[0635] The server inputs the received image data into an analysis model using TensorFlow and performs image analysis. The input is the transmitted image data, and the output is the analysis results regarding the damage features. Here, feature extraction is performed using deep learning techniques.

[0636] Step 4:

[0637] The server compares the analysis results with historical data records and a MySQL database to determine if there is damage. The input consists of the analysis results and historical data records, and the output is a determination of whether or not there is damage. This determination determines whether the item is normal or damaged.

[0638] Step 5:

[0639] The server automatically generates a report based on the assessment result and notifies the responsible person. The input is the assessment result, and the output is the generated report and the notification. The report includes details of the damage and whether compensation is required, and is designed to reach the responsible person quickly.

[0640] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0641] This invention relates to a system that integrates user emotion recognition technology into the damage assessment process when a device is returned. The system includes a device, a server, a communication network, an image analysis model, and an emotion engine.

[0642] When a user returns a device, the device's camera captures a detailed image of the device's appearance upon return. This image data is transmitted to a server via the network. Simultaneously, an emotion engine built into the device analyzes the user's facial expressions and voice, and sends the results to the server. The emotion engine has the capability to accurately determine whether the user is nervous or relieved upon returning the device.

[0643] The server analyzes the received image data using a modifiable deep learning model. The deep learning model identifies damage characteristics based on a vast amount of historical data and compares them to previous damage records. User sentiment data can also be used in the analysis, allowing the judgment method to be adjusted according to the user's state.

[0644] Based on the analysis results, the server determines the extent of damage to the returned device and generates a report. This report is presented in an appropriate format, taking into account the user's emotional state. For example, if the user is stressed when returning the device, the notification content is adjusted to be gentler and more reassuring.

[0645] As a concrete example, consider a case where a user returns a device with a small crack on the screen. If the emotion engine analyzes that the user is showing anxiety or tension regarding this situation, the server will send a notification that includes a sincere response in addition to the usual repair procedures. This will reassure the user, and the problem will be resolved.

[0646] This system not only streamlines the process of determining damage when devices are returned, but also improves the user experience.

[0647] The following describes the processing flow.

[0648] Step 1:

[0649] The user returns the device. The return system is activated when the user acknowledges that they have finished using the device and brings it to the designated return location.

[0650] Step 2:

[0651] The device takes pictures of its exterior. Upon receiving the device, it uses its built-in camera to take pictures of its exterior from multiple angles. During this process, the device automatically checks the image quality and retakes the pictures if necessary.

[0652] Step 3:

[0653] The device activates the emotion engine. The emotion engine records the user's facial expressions and voice, and analyzes the user's emotional state. For example, it can determine in real time whether the user is smiling or feeling nervous.

[0654] Step 4:

[0655] The device sends images and emotion data captured by the device to the server. Through the communication network, the device uploads high-resolution image data along with the analyzed user emotion data to the server.

[0656] Step 5:

[0657] The server analyzes the image data. An AI analysis model installed on the server uses deep learning technology to analyze the images and extract the characteristics of the damage that occurred on the device.

[0658] Step 6:

[0659] The server incorporates emotional data into its analysis. The server uses the user's emotional data to adjust the damage assessment and make more appropriate judgments.

[0660] Step 7:

[0661] The server performs damage assessment and generates a report. Based on the analysis results, it determines whether the terminal's condition is due to intentional damage or normal wear and tear, and generates a detailed report.

[0662] Step 8:

[0663] The server notifies the user. Considering the report's contents, it creates an appropriate notification message tailored to the user's emotional state and sends it to the user via email, app notifications, or other means.

[0664] (Example 2)

[0665] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] Conventional equipment return damage assessment processes merely assessed and reported only the physical damage to the equipment, lacking consideration for the user's psychological state. As a result, users were not provided with adequate reassurance or support, sometimes leading to a poor user experience. This invention aims to improve the user experience by considering the user's emotions during the return process and providing more appropriate assessment and reporting.

[0667] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0668] In this invention, the server includes means for acquiring return information after use and user emotional information, means for analyzing the accumulated image data and emotional information, and means for reporting the damage status of the equipment based on the analysis results and notifying the user in a format that takes into account their emotional state. This enables comprehensive judgment and reporting that takes into account not only physical damage but also the user's psychological state.

[0669] "Return information" refers to data acquired when a user returns equipment, and it indicates the status of the return and its physical condition.

[0670] "Emotional information" refers to data that indicates the user's psychological state, extracted from their facial expressions and voice, and is used to identify the user's emotions at the time of return.

[0671] A "communication network" is a system that exchanges data via various wired or wireless technologies, using paths and media for transmitting information.

[0672] An "analysis algorithm" is a procedure or method for processing large amounts of data and extracting or determining information according to a specific purpose.

[0673] A "machine learning algorithm" is a set of methods that automatically learn from past data and experience to perform identification and prediction, and is used to recognize patterns in data.

[0674] "Damage assessment" refers to the process of identifying and evaluating whether there is any physical or functional damage to equipment, and to what extent.

[0675] "Notification" refers to the act of conveying a decision or information to a user, and involves sending an appropriate message depending on the situation.

[0676] This system is an integrated system designed to take into account the user's emotions when equipment is returned, enabling more appropriate damage assessment and reporting. It mainly consists of a combination of terminals, servers, communication networks, data analysis algorithms, and an emotion analysis engine.

[0677] The device automatically activates its camera when the user returns the equipment, taking a picture of the equipment's appearance upon return. In addition, it records the user's emotional information using facial recognition and voice analysis technologies. This includes common computer vision techniques for face detection and software for voice analysis. This information is transmitted to a server using wireless communication technologies (e.g., Wi-Fi or mobile networks).

[0678] The server processes the received image data and sentiment data. The image data is analyzed using machine learning algorithms to identify corruption features. It is preferable to use deep learning frameworks such as TensorFlow or PyTorch for the analysis algorithms. The analysis results are compared with historical databases, and if new corruption is detected, its details are reflected in the report.

[0679] Emotional data is used to determine the user's psychological state. The emotion analysis engine analyzes the emotions the user is experiencing at the time of return based on collected facial and voice data. The resulting emotional state is used to adjust the content and tone of the report. For example, if the user is feeling anxious, the report will be written in a more considerate manner, and additional reassuring information will be provided.

[0680] As a concrete example, consider a case where a user returns a smartphone with a small crack in the screen. If the emotion analysis engine determines that the user appears anxious, the server will generate a reassuring notification in addition to the usual damage assessment. This allows the user to feel that they are receiving a sincere response and to have confidence that the problem will be resolved.

[0681] Example of a prompt:

[0682] "Please explain how you will analyze the emotions users experience when returning their devices and how you will adjust the damage assessment process based on those results."

[0683] This system enables efficient and highly satisfying service delivery through a comprehensive approach that considers not only physical damage but also the user's psychological state.

[0684] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0685] Step 1:

[0686] The device automatically activates its built-in camera upon return and takes a picture of the device's exterior. The captured image data is then processed, including exposure correction and noise reduction, before being sent to the server. The input is the physical exterior of the device, and the output is the processed image data.

[0687] Step 2:

[0688] The device records the user's facial expressions and voice, and analyzes them in real time using an emotion analysis engine. This analysis utilizes facial recognition technology and voice feature analysis to identify the user's emotional state (e.g., tension, relief, anxiety). Input consists of the user's facial image and voice data, and output is the determined emotion information.

[0689] Step 3:

[0690] The server feeds the received image data into a machine learning algorithm to extract damage features. This process uses a conventional deep learning model and compares the data against a historical database. The input is image data, and the output is information indicating the location and characteristics of the damage.

[0691] Step 4:

[0692] The server analyzes emotional information and evaluates the user's psychological state. Based on this information, it adjusts the criteria for damage assessment to perform a more accurate evaluation. The input is emotional information, and the output is the adjusted damage assessment criteria.

[0693] Step 5:

[0694] The server generates a report on the equipment's damage status based on the integrated analysis results. Since the content and tone of this report are adjusted according to the user's emotional state, the inputs are damage information and emotional information, and the output is a customized report for the user.

[0695] (Application Example 2)

[0696] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0697] There is a need for a system that simultaneously improves damage assessment and user experience when returning consumer robots. In particular, the challenge is to enable flexible responses that take into account the user's emotions and reduce stress during the return process.

[0698] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0699] In this invention, the server includes means for acquiring return information after use, means for photographing the appearance of the equipment, means for accumulating the captured image data via a communication network, means for analyzing the accumulated image data using an analysis model, means for analyzing the user's emotions, and means for adaptively adjusting the report content based on the results of the emotion analysis means. This not only enables efficient determination of equipment damage upon return but also allows for flexible responses in accordance with the user's psychological state.

[0700] "Means for obtaining return information after use" refers to a device or method that has the function of obtaining information regarding the return of equipment when its use has ended.

[0701] "Means for photographing the appearance of equipment" refers to a device or method for photographing the appearance of equipment as an image in order to record the surface condition and characteristics of the equipment.

[0702] "Means of aggregating information via a communication network" refers to a device or method that has the function of collecting information through a network into a centralized database or server.

[0703] An "analysis model" is a computational model built based on past data and learning results in order to analyze acquired data.

[0704] "Emotional analysis means" refers to a device or method for detecting and analyzing emotions from a user's facial expressions, voice, etc.

[0705] "Means of adaptive adjustment" refers to a device or method that has the function of automatically changing the optimal response according to the situation based on the analysis results.

[0706] The system for carrying out this invention mainly includes a server, a terminal, a communication network, a camera, and an emotion analysis engine. The terminal is equipped with a camera that acquires return information after use and also takes pictures of the device's appearance. The captured image data is transmitted to the server via the communication network.

[0707] After receiving this image data, the server analyzes the data using an analysis model. This analysis model is built on a deep learning algorithm and determines equipment damage by comparing past data records with current image data.

[0708] Furthermore, the emotion analysis engine installed in the device analyzes the user's emotions from their facial expressions and voice. This emotional data is sent to a server and reflected in the damage assessment report. Based on this information, the server adaptively adjusts the report content according to the user's emotions and notifies the user. This notification is designed to be flexible and considerate using a generative AI model.

[0709] For example, if a user returns equipment and notices minor cosmetic damage, and the user expresses tension or anxiety, the system will generate a notification that uses gentle language to explain the situation and provide reassurance.

[0710] An example of a prompt message is, "Generate thoughtful feedback based on the user's feelings and the device's condition upon return."

[0711] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0712] Step 1:

[0713] The terminal retrieves return information and uses its camera to photograph the exterior of the returned equipment. The input is the returned equipment, and the output is an image of the equipment's exterior. The terminal uses its camera to take detailed photographs of the entire front surface of the equipment.

[0714] Step 2:

[0715] The device transmits image data it captures to a server via a communication network. The input is an image of the device's exterior, and the output is the image data transmitted to the server. The device performs appropriate compression and encryption to securely upload the data to the server.

[0716] Step 3:

[0717] The server inputs the received image data into an analysis model and analyzes the data. The input is the image data sent to the server, and the output is a judgment result indicating the presence and extent of damage. The server utilizes a deep learning algorithm to identify damage characteristics by comparing them with past data.

[0718] Step 4:

[0719] The device detects the user's facial expressions and voice, and analyzes those emotions using an emotion analysis engine. The input is the user's facial image and voice data, and the output is data indicating the user's emotional state. The device performs emotion analysis in real time, determining things like tension and a sense of security.

[0720] Step 5:

[0721] The server receives emotion analysis data and generates a damage assessment report based on it. The input is the damage assessment result and the user's emotion data, and the output is the report content to be notified to the user. The server uses a generation AI model and utilizes prompt text to generate feedback content.

[0722] Step 6:

[0723] The user receives the report and chooses further actions based on it. The input is the report information notified to the user, and the output is the user's next action or feedback. The user can proceed with the return process with confidence.

[0724] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0725] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0726] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0727] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0728] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0729] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0730] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0731] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0732] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0733] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0734] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0735] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0736] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0737] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0738] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0739] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0740] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0741] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0742] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0743] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0744] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0745] The following is further disclosed regarding the embodiments described above.

[0746] (Claim 1)

[0747] A means of obtaining return information after use,

[0748] A means for photographing the exterior of the terminal based on the aforementioned return information,

[0749] A means for collecting the captured image data via a communication network,

[0750] A means of analyzing the accumulated image data using an analysis model,

[0751] A means for comparing the results of the analysis model with past data records to determine damage,

[0752] A means of reporting the status of the returned device based on the judgment result,

[0753] A system that includes this.

[0754] (Claim 2)

[0755] The system according to claim 1, wherein the analysis model extracts damage features using a deep learning algorithm.

[0756] (Claim 3)

[0757] The system according to claim 1, further comprising means for automatically determining whether or not to make a payment request and for providing notification.

[0758] "Example 1"

[0759] (Claim 1)

[0760] A means of obtaining information regarding the return of the product after use,

[0761] An imaging means for capturing images of the appearance of an information terminal from multiple viewpoints based on the aforementioned return information,

[0762] A means for storing the captured image data via an information and communication network,

[0763] A means of analyzing accumulated image data using an analysis model,

[0764] A means of determining damage by comparing the output of the analysis model with past information records,

[0765] A means for reporting the status of the information terminal returned based on the judgment result,

[0766] A means of notifying users of the report results,

[0767] A system that includes this.

[0768] (Claim 2)

[0769] The system according to claim 1, wherein the analysis model extracts damage features using a machine learning algorithm.

[0770] (Claim 3)

[0771] The system according to claim 1, further comprising means for automatically determining and notifying a claim for damages based on whether or not damage has occurred.

[0772] "Application Example 1"

[0773] (Claim 1)

[0774] A means of obtaining return information after use,

[0775] A means for photographing the exterior based on the aforementioned return information,

[0776] A means for collecting the captured image data via a communication infrastructure,

[0777] A means of analyzing the accumulated image data using an analysis model,

[0778] A means for comparing the results of the analysis model with past data records to determine damage,

[0779] A means of reporting the status of the returned equipment based on the judgment result,

[0780] A means of determining compensation based on the damage results and having the person in charge prepare a report,

[0781] A system that includes this.

[0782] (Claim 2)

[0783] The system according to claim 1, wherein the analysis model extracts damage features using a machine learning algorithm.

[0784] (Claim 3)

[0785] The system according to claim 1, further comprising means for automatically determining whether or not to make a compensation claim and for providing notification.

[0786] "Example 2 of combining an emotion engine"

[0787] (Claim 1)

[0788] A means of obtaining return information after use and user sentiment information,

[0789] A means for photographing the appearance of the equipment based on the aforementioned return information,

[0790] A means for collecting the aforementioned captured image data and emotional information via a communication network,

[0791] A means for analyzing the accumulated image data using an analysis algorithm,

[0792] A means for analyzing accumulated emotional information and adjusting judgment criteria based on the user's emotional state,

[0793] A means for comparing the results of the analysis algorithm with past information records to determine damage,

[0794] Based on the assessment results, a means of reporting the status of the returned equipment and notifying the user in a format that takes into account their emotional state,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, wherein the analysis algorithm extracts damage features using a machine learning algorithm and optimizes the judgment process based on emotional information.

[0798] (Claim 3)

[0799] The system according to claim 1, further comprising means for automatically determining whether or not to request payment and for notifying the user while taking into account the user's emotional state.

[0800] "Application example 2 when combining with an emotional engine"

[0801] (Claim 1)

[0802] A means of obtaining return information after use,

[0803] A means for photographing the appearance of the equipment based on the aforementioned return information,

[0804] A means for collecting the captured image data via a communication network,

[0805] A means of analyzing the accumulated image data using an analysis model,

[0806] A means for comparing the results of the analysis model with past data records to determine damage,

[0807] A means of reporting the status of the returned equipment based on the judgment result,

[0808] An emotion analysis tool for analyzing the user's emotions,

[0809] Means for adaptively adjusting the report content based on the results of the aforementioned emotion analysis means,

[0810] A system that includes this.

[0811] (Claim 2)

[0812] The system according to claim 1, wherein the analysis model extracts damage features using a hierarchical learning algorithm.

[0813] (Claim 3)

[0814] The system according to claim 1, further comprising means for automatically determining whether or not to request repayment and providing notification, and adjusting the content of the notification based on the user's emotional data. [Explanation of Symbols]

[0815] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining return information after use, A means for photographing the exterior of the terminal based on the aforementioned return information, A means for collecting the captured image data via a communication network, A means of analyzing the accumulated image data using an analysis model, A means for comparing the results of the analysis model with past data records to determine damage, A means of reporting the status of the returned device based on the judgment result, A system that includes this.

2. The system according to claim 1, wherein the analysis model extracts damage features using a deep learning algorithm.

3. The system according to claim 1, further comprising means for automatically determining whether or not to make a payment request and for providing notification.

Images