System

Abstract

A system is provided.SOLUTION: The system includes a means for collecting reviews from users, a means for preprocessing the collected reviews and inputting them to an evaluation model, a means for generating the evaluation model of delivery by using generation artificial intelligence, a means for evaluating new delivery data by using the evaluation model and outputting an evaluation result, and a means for feeding back the evaluation result and improving the performance of a deliverer or a transport company.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional delivery systems have the problem that it is difficult to evaluate delivery quality and efficiency, making it difficult to identify areas for improvement. Furthermore, there is no mechanism for quantitatively evaluating the performance of individual delivery personnel, and no method for systematically utilizing user feedback. Furthermore, there is a lack of mechanisms for allowing delivery personnel to flexibly set delivery fees, or preferential treatment of fees based on evaluation. Given these circumstances, there is a need for improved delivery quality and the creation of an efficient delivery system. [Means for solving the problem]

[0005] This invention provides a means for collecting user reviews, preprocessing them, and inputting them into a rating model. Furthermore, we have developed a system that uses generative artificial intelligence to generate a delivery rating model, evaluates new delivery data, and outputs the results. These evaluation results are fed back to delivery personnel and transportation companies to encourage performance improvements. The flexibility and efficiency of the delivery system are also improved by adding a means for delivery personnel to set delivery fees based on the evaluation results, a means for automatically determining upper limits on delivery fees, and a means for offering preferential fees to delivery personnel with a certain rating or higher based on the evaluation results. Furthermore, a sustainable business model is created by registering and managing individual business owners and collecting fees for using the system.

[0006] "User" refers to an individual or corporation that uses the delivery service.

[0007] A "review" is text data that contains a user's evaluation or opinion of a delivery service.

[0008] "Preprocessing" refers to processes such as data cleansing and normalization to convert collected data into an analyzable format.

[0009] An "evaluation model" is an analytical model created using generative artificial intelligence to evaluate delivery quality and efficiency.

[0010] "Generative AI" refers to artificial intelligence that generates models based on data using machine learning and deep learning technologies.

[0011] "Delivery Data" refers to a data set containing detailed information about deliveries, and is the data that is input into the valuation model.

[0012] "Evaluation results" refer to the evaluation scores and improvement suggestions regarding delivery quality and efficiency calculated by the evaluation model.

[0013] "Feedback" refers to the process of notifying delivery personnel and transportation companies of the evaluation results and suggesting areas for improvement.

[0014] A "delivery person" is an individual or corporation whose job is to deliver packages to customers.

[0015] A "transportation company" is a company that employs multiple delivery personnel and provides delivery services.

[0016] "Delivery Fee" means the fee paid by the Customer for the provision of delivery services.

[0017] An "independent contractor" is an individual who operates independently for the delivery services that he or she provides.

[0018] "Fees" means fees collected for system use or service provision.

[0019] "Registration management" refers to the process of managing the registration procedures for system users and sole proprietors.

[0020] "Transmission" is the process of conveying data or notifications to other terminals or systems.

[0021] A "terminal" refers to an electronic device such as a smartphone or tablet used by a delivery person or user. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention aims to improve delivery quality and efficiency by introducing a delivery evaluation system. Specific embodiments for carrying out the present invention will be described below.

[0044] This system consists of a user, a terminal, and a server. The user uses the delivery service and then rates it. Delivery personnel and transportation companies use the terminal to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates rating models, performs the rating, and provides feedback on the results.

[0045] Program processing overview

[0046] 1. Data Collection

[0047] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[0048] The server collects reviews in real time and stores them in a database.

[0049] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0050] 2. Data Preprocessing

[0051] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[0052] 3. Generating the evaluation model

[0053] The server creates a machine learning dataset based on the preprocessed data.

[0054] The server uses artificial intelligence to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[0055] 4. Conducting the evaluation

[0056] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[0057] 5. Feedback of evaluation results

[0058] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[0059] 6. Expanding system usage

[0060] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[0061] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0062] 7. Monetization

[0063] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[0064] Specific examples

[0065] Processing user reviews

[0066] 1. A user actually uses the delivery service and, after the delivery is completed, posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[0067] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[0068] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[0069] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[0070] Determining shipping fees

[0071] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[0072] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions.

[0073] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and calculates the optimal shipping fee.

[0074] This specific form of the present invention enables systematic evaluation and improvement of delivery quality and efficiency, and also realizes flexible pricing and preferential handling fees based on the evaluation, which is expected to optimize the entire delivery service.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] A user uses a delivery service and writes a review after delivery is completed.

[0078] Users post reviews and ratings about deliveries using a mobile app or web interface.

[0079] Step 2:

[0080] The server collects user reviews.

[0081] The server stores the reviews in a database in real time.

[0082] Step 3:

[0083] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0084] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[0085] Step 4:

[0086] The server preprocesses the collected data.

[0087] The server normalizes the review text data, removes invalid data, and imputes missing values.

[0088] Step 5:

[0089] The server generates a dataset for machine learning.

[0090] The server extracts features and labels them based on the preprocessed data.

[0091] Step 6:

[0092] The server trains the evaluation model using generative AI.

[0093] The server uses machine learning algorithms to train a rating model for deliveries.

[0094] Step 7:

[0095] The server inputs the new delivery data into the rating model.

[0096] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[0097] Step 8:

[0098] The server generates the evaluation results.

[0099] The server generates evaluation results and improvement suggestions based on the evaluation scores output from the model.

[0100] Step 9:

[0101] The server notifies the delivery person and the transportation company of the evaluation results.

[0102] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[0103] Step 10:

[0104] The terminal displays the evaluation results.

[0105] The terminal (delivery person's smartphone or tablet) displays the evaluation results and improvement suggestions.

[0106] Step 11:

[0107] The terminal provides the delivery person with the option to set delivery fees.

[0108] The terminal displays a delivery fee setting screen to the delivery person, allowing the delivery person to set the delivery fee.

[0109] Step 12:

[0110] The server automatically determines the shipping limit.

[0111] The server automatically sets an appropriate upper limit on delivery charges, taking into account delivery conditions and the delivery person's past evaluations.

[0112] Step 13:

[0113] The delivery person sets the shipping fee.

[0114] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[0115] Step 14:

[0116] The server manages the registration of sole proprietors.

[0117] The server accepts the registration information of the sole proprietor and checks the qualifications based on the evaluation criteria.

[0118] Step 15:

[0119] The server calculates and collects fees for using the system.

[0120] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[0121] Example 1

[0122] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] Current delivery services lack a system that accurately evaluates delivery quality and efficiency and provides appropriate feedback to delivery personnel and transportation companies. As a result, it is difficult to improve service quality and efficiency, and user satisfaction declines. In addition, there is a lack of a system that effectively sets appropriate fees and provides preferential treatment for commissions based on evaluations, making it difficult to improve delivery personnel motivation or ensure consistency in service quality.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0125] In this invention, the server includes a means for collecting reviews from users, a means for preprocessing the collected reviews and converting them into a format suitable for evaluation using a natural language processing tool, and a means for creating a machine learning dataset based on the preprocessed data, which enables accurate evaluation of delivery quality and efficiency and provides appropriate feedback.

[0126] "User" refers to a general consumer who uses goods or services.

[0127] "Review" refers to the evaluation or feedback provided by a user after using a product or service.

[0128] "Preprocessing" refers to the process of preparing collected data in a form that can be analyzed and evaluated.

[0129] "Natural language processing tools" refer to software technology for analyzing text data and extracting necessary information.

[0130] A "machine learning dataset" refers to a collection of data used to train a machine learning model.

[0131] "Generative artificial intelligence" refers to programs that have the ability to perform specific tasks using machine learning and deep learning.

[0132] An "evaluation model" refers to an algorithm or system that analyzes data and produces results based on specific evaluation criteria.

[0133] "Delivery Data" means data containing information regarding the progress and results of a delivery.

[0134] "Evaluation Results" refers to the numerical or qualitative evaluations obtained using an evaluation model.

[0135] "Feedback" refers to the process of informing stakeholders of evaluation results and areas for improvement.

[0136] "Fees" refers to the fees paid for use of the Services.

[0137] "Delivery Charge" means the charge payable for delivery of Goods.

[0138] "Maximum shipping fee" refers to the maximum shipping fee set based on the shipping conditions.

[0139] "Sole proprietor" refers to an individual who operates a business as a self-employed person.

[0140] "Evaluation criteria" refers to the standards and indicators used for evaluation.

[0141] "Qualifications" refers to the abilities and conditions necessary to perform a particular role or task.

[0142] The purpose of this invention is to improve delivery quality and efficiency by introducing a delivery evaluation system. This system consists of three entities: a user, a terminal, and a server.

[0143] 1. Data Collection

[0144] After the delivery is complete, the user can leave a review via a mobile app or web interface. For example, the user can write a comment such as "The delivery was 30 minutes later than expected."

[0145] The server collects these reviews in real time and stores them in a database (e.g., MySQL or PostgreSQL). The server is configured to receive and store data using a REST API. The server also collects information about damage and delays from the terminals of delivery companies and delivery personnel. This process is carried out via API, and the latest information is maintained by periodic data transmission from the terminals.

[0146] 2. Data Preprocessing

[0147] The server preprocesses the collected data. Specifically, it analyzes the collected text data using natural language processing tools (e.g., NLTK or SpaCy). This analysis generates the keywords and scoring necessary for evaluation. The server also removes invalid data and completes missing values. For example, invalid data would be data that has clearly been entered incorrectly or is in a non-standard format, and this is eliminated through automatic screening.

[0148] 3. Generating the evaluation model

[0149] The server creates a machine learning dataset based on the preprocessed data. The dataset includes features such as delivery time, rating points, and text reviews. The server then generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch). This model has the ability to train the dataset and evaluate delivery quality and efficiency. The generated model is stored internally on the server for use during evaluation.

[0150] 4. Conducting the evaluation

[0151] The server inputs new delivery data into the evaluation model and calculates the evaluation results. For example, when new delivery data is input into the model, an evaluation score and qualitative comments are automatically generated. The server saves the evaluation results in a database and simultaneously adds them to a notification queue. This process makes the evaluation results available immediately.

[0152] 5. Feedback of evaluation results

[0153] The server notifies the delivery person or transportation company of the evaluation results. This notification is sent instantly using an API or WebSocket. The device receives the notification and displays the evaluation results and areas for improvement to the delivery person. For example, specific feedback such as "Please be punctual" may be included.

[0154] 6. Expanding system usage

[0155] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. The delivery person adjusts the delivery fee based on the evaluation using the terminal UI. The server manages the registration of individual business owners and checks their eligibility based on the evaluation criteria. For example, a warning message is displayed if the evaluation criteria are not met during new registration.

[0156] 7. Monetization

[0157] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This is done using payment gateways such as credit cards and bank transfers. The collected fee is transferred to the operator's account and used to cover system maintenance costs.

[0158] Specific examples

[0159] Processing user reviews

[0160] 1. After completing delivery, the user posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[0161] 2. The server stores this review in a database, analyzes the text (e.g., SpaCy), and scores it as "delayed."

[0162] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[0163] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[0164] Determining shipping fees

[0165] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[0166] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions. For example, the system may display a message saying, "We recommend setting the fee between 1,000 and 1,200 yen."

[0167] 3. The server sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and calculates the optimal delivery fee. For example, dynamic pricing is possible, such as setting an additional fee for late-night deliveries.

[0168] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0169] Step 1:

[0170] The user writes a review after the delivery is completed.

[0171] Specifically, users use a mobile app or web interface to post comments such as "Delivery was 30 minutes later than expected."

[0172] Input: User-entered text review

[0173] Output: Text review data sent to the server

[0174] Step 2:

[0175] The server receives reviews from users and stores them in a database in real time.

[0176] Specifically, the server receives data via a REST API and stores it in a MySQL or PostgreSQL database, validating the data format during this process.

[0177] Input: User-submitted text review

[0178] Output: Review information stored in a database

[0179] Step 3:

[0180] The server preprocesses the collected data.

[0181] Specifically, we use natural language processing tools (e.g., NLTK and SpaCy) to analyze text data and convert it into a format suitable for evaluation. We also remove invalid data and impute missing values.

[0182] Input: Review information stored in the database

[0183] Output: Preprocessed text data

[0184] Step 4:

[0185] The server creates a machine learning dataset based on the preprocessed data.

[0186] Specifically, collected features such as review information, delivery time, and rating points are incorporated into a dataset, which generates a dataset in a format that can be used for rating model training.

[0187] Input: Preprocessed text data

[0188] Output: Dataset for machine learning

[0189] Step 5:

[0190] The server generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch).

[0191] Specifically, the system uses preprocessed data to train a model and generate an evaluation model capable of evaluating delivery quality and efficiency. The generated model is stored on the server.

[0192] Input: Machine learning dataset

[0193] Output: Evaluation model

[0194] Step 6:

[0195] The server inputs new delivery data into the rating model, calculates and outputs the rating results.

[0196] Specifically, new delivery data (e.g., delivery time, user ratings) is input into the evaluation model, and evaluation scores and improvement suggestions are automatically generated.

[0197] Input: New shipping data

[0198] Output: Evaluation results (score, improvement suggestions)

[0199] Step 7:

[0200] The server notifies the delivery person or transportation company of the evaluation results.

[0201] Specifically, the evaluation results are instantly sent via API and WebSocket and displayed on the device, including not only the evaluation score but also points for improvement.

[0202] Input: Evaluation result

[0203] Output: Evaluation results displayed on the terminal of the delivery person or transportation company

[0204] Step 8:

[0205] The terminal provides a shipping fee setting option, and the delivery person sets the shipping fee.

[0206] Specifically, a pricing screen based on AI suggestions is displayed on the device's UI, and the delivery person inputs the optimal delivery fee. Based on this, the server calculates the appropriate fee and sets an upper limit.

[0207] Input: Shipping fee information set on the device

[0208] Output: Shipping fee settings reflected in the system

[0209] Step 9:

[0210] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0211] Specifically, if the evaluation criteria are not met during new registration, a warning message will be displayed. The server will check the qualifications of the sole proprietor based on the evaluation score and decide whether to approve the registration.

[0212] Input: New registration information

[0213] Output: Credential check results and warning messages (if necessary)

[0214] Step 10:

[0215] The server calculates and collects the system usage fee as a percentage of the shipping fee.

[0216] Specifically, fees are automatically collected from transportation companies and individual business owners through a credit card payment gateway and transferred to the operator's account.

[0217] Input: Shipping fee information, handling fee percentage

[0218] Output: Collected fees and transfer details

[0219] (Application example 1)

[0220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0221] In conventional food delivery services, evaluation of delivery quality and efficiency is subjective, and areas for improvement are not clearly communicated, making it difficult to improve overall service quality. Furthermore, delivery fees are set based on the delivery person's experience and intuition, which can lead to inappropriate pricing, resulting in reduced revenue and customer satisfaction. Furthermore, evaluation criteria are vague, making it difficult for delivery people and transportation companies to obtain useful feedback to optimize their performance. These issues need to be resolved.

[0222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0223] In this invention, the server includes a means for collecting user reviews, a means for preprocessing the collected reviews and inputting them into a rating model, and a means for generating a delivery rating model using generative artificial intelligence. This enables objective evaluation of delivery quality and efficiency. The server further includes a means for inputting new delivery data into the rating model and outputting the rating results, a means for notifying the user of the rating results and providing delivery fee setting options, and a means for providing feedback on the rating results and suggesting areas for performance improvement. This allows delivery personnel and transportation companies to objectively evaluate their performance and identify specific areas for improvement based on the feedback. The server also includes a means for automatically determining a maximum delivery fee based on delivery conditions, a means for offering preferential commissions to delivery personnel who receive a certain rating or higher based on the rating results, a means for evaluating delivery personnel based on delivery data from the food delivery service, and a means for providing feedback on specific areas for improvement to delivery personnel who do not meet certain rating standards. This enables appropriate delivery fee settings and fair preferential commissions, which is expected to improve the overall quality of the service.

[0224] "User" refers to a person who uses the food delivery service to order and receive meals.

[0225] "Reviews" refer to opinions and impressions provided by users after delivery is completed.

[0226] "Preprocessing" refers to the process of analyzing collected reviews and converting them into a format suitable for the rating model.

[0227] An "evaluation model" refers to a machine learning model that has criteria for evaluating delivery quality and efficiency.

[0228] "Delivery Data" refers to all data related to delivery (delivery time, delay information, user reviews, etc.).

[0229] "Evaluation results" refer to the numerical values ​​and qualitative indicators output by the evaluation model that indicate the quality and efficiency of delivery.

[0230] "Feedback" refers to the points for improvement and evaluation provided based on the evaluation results.

[0231] "Performance" refers to the efficiency and quality of delivery work performed by delivery personnel and transportation companies.

[0232] "Delivery fee" refers to the fee paid by the user when using the delivery service.

[0233] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[0234] "Food delivery service" refers to a service that allows users to order food and drinks online and have them delivered to a specified location.

[0235] "Generative AI" refers to AI that uses machine learning and data analysis to perform specific tasks.

[0236] The present invention aims to evaluate delivery quality and efficiency in food delivery services and improve the overall quality of the service. Specific embodiments for carrying out the present invention will be described below.

[0237] System Configuration

[0238] The system of the present invention is mainly composed of a server, a terminal (such as a smartphone), and a user.

[0239] server:

[0240] The server is the core component that collects data, preprocesses it, generates evaluation models, performs the evaluation, and provides feedback on the results. Specific software used includes Python, Pandas, and Scikit-learn.

[0241] Device:

[0242] The terminal is a device used by delivery personnel and transportation companies to check evaluation results, set delivery fees, and receive feedback. Specifically, this is realized as a smartphone app.

[0243] User:

[0244] Users are people who use food delivery services and post reviews after delivery is complete.

[0245] System Features

[0246] 1. Data Collection:

[0247] When a user uses a food delivery service and the delivery is completed, the user enters a review through the mobile app. The server collects these reviews in real time and stores them in a database.

[0248] 2. Data Preprocessing:

[0249] The server preprocesses the collected data. It analyzes the review text data and converts it into a format suitable for the rating model. For example, if a review says "30 minutes late," it will be scored as "late."

[0250] 3. Generate the evaluation model:

[0251] The server creates a machine learning dataset based on the preprocessed data, generates a random forest model using Scikit-learn, and builds a delivery evaluation model.

[0252] 4. Conducting the evaluation:

[0253] New delivery data is input into the evaluation model, and the evaluation results are calculated. The evaluation results are output as numerical or qualitative evaluations, and the server stores these results.

[0254] 5. Feedback of evaluation results:

[0255] The server then notifies the delivery person or the transportation company of the evaluation results. For example, the delivery person's device may receive feedback such as, "Next time, try to be more punctual."

[0256] 6. Set up shipping rates:

[0257] Based on the evaluation results, the delivery person sets the delivery fee on their smartphone. The server calculates the upper limit of the delivery fee based on delivery conditions (distance, time zone, etc.) and proposes the optimal delivery fee.

[0258] Specific examples

[0259] For example, if a user posts a review on a mobile app stating that "the delivery was 30 minutes later than scheduled," the server stores this review in a database and uses text analysis to score it as "delayed." This data is then input into a rating model, which rates the delivery person's performance as "3.5 / 5." This rating result is sent to the delivery person's smartphone, and feedback such as "Please be punctual next time" is displayed. The delivery person can also adjust the delivery fee based on this result.

[0260] Example prompt sentence:

[0261] "User review: 30 minutes late. Please analyze the cause of the delivery person's delay and provide a rating."

[0262] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0263] Step 1: User submits review after delivery is complete.

[0264] Input: A review a user leaves on a food delivery app (e.g., "Delivery was 30 minutes later than expected").

[0265] Output: User submitted reviews are sent to the server through the app and stored in a database.

[0266] Specific operation: The user uses the smartphone app to write down their thoughts and opinions after the delivery is completed and presses the submit button. The app then sends the review to the server, which records the review in a database.

[0267] Step 2: The server preprocesses the collected reviews.

[0268] Input: User reviews collected in step 1.

[0269] Output: The text data is converted into a format suitable for the evaluation model (e.g., scored as "delay").

[0270] How it works: The server uses Python and Pandas to parse the review text data, checking for the presence of keywords (e.g., "delay"), and then converts the results into scores in a format suitable for the rating model.

[0271] Step 3: The server generates the evaluation model.

[0272] Input: A preprocessed dataset (e.g., a dataset of "delay" scores).

[0273] Output: A delivery evaluation model (e.g., a random forest model).

[0274] How it works: The server uses Scikit-learn to generate an evaluation model, such as a random forest model, from the preprocessed dataset. The model has criteria for evaluating delivery quality and efficiency.

[0275] Step 4: The server inputs the new delivery data into the rating model and performs the rating.

[0276] Input: New delivery data (e.g., new user reviews and their preprocessing results).

[0277] Output: Evaluation result (e.g., delivery driver performance score "3.5 / 5").

[0278] Specific operation: The server inputs new delivery data into the evaluation model to evaluate the delivery person's performance. The evaluation results are generated as numerical values ​​and stored in the database.

[0279] Step 5: The server notifies the delivery person's terminal of the evaluation result.

[0280] Input: Evaluation result (e.g., performance score "3.5 / 5" with specific feedback).

[0281] Output: A notification is sent to the delivery person's smartphone (e.g., "Please be punctual next time").

[0282] Specific operation: The server compiles the evaluation results and generates a notification message. It then sends the message to the delivery person through a smartphone app. The delivery person receives the notification and checks the feedback on the app.

[0283] Step 6: The delivery person sets the shipping fee based on the evaluation results.

[0284] Input: Evaluation results and AI-generated shipping fee suggestions.

[0285] Output: New shipping price configuration (e.g. suggested optimal shipping price).

[0286] Specific operation: The delivery person opens the delivery fee setting screen on their smartphone app and determines the fee based on the AI's suggestions. The server calculates the maximum delivery fee based on delivery conditions (distance, time zone, etc.) and displays the optimal delivery fee. The delivery person sets the delivery fee according to the suggestions and sends it to the server.

[0287] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0288] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. Below, we will explain in detail how each component works together to realize the present invention.

[0289] This system consists of a user, a terminal, and a server. Users use delivery services and then rate them. Delivery personnel and shipping companies use their terminals to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates a rating model, performs the rating, and provides feedback on the results. Furthermore, an emotion engine is used to recognize emotions from user reviews and use them as part of the rating.

[0290] Program processing overview

[0291] 1. Data Collection

[0292] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[0293] The server collects reviews in real time and stores them in a database.

[0294] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0295] 2. Data Preprocessing

[0296] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[0297] 3. Emotion recognition

[0298] The server uses an emotion engine to recognize the user's emotion (e.g., satisfaction, dissatisfaction, surprise, etc.) from the review text data.

[0299] The emotion engine categorizes and scores the emotional data it recognizes.

[0300] 4. Generating the evaluation model

[0301] The server creates a machine learning dataset based on the preprocessed data and emotion data.

[0302] The server uses generative AI to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[0303] 5. Conducting the evaluation

[0304] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[0305] 6. Feedback of evaluation results

[0306] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[0307] 7. Emotional Data Feedback

[0308] The server provides the delivery person with the user's emotional data recognized by the emotion engine as feedback, allowing the delivery person to understand the user's specific emotional state and take measures to improve the situation.

[0309] 8. Expanding System Usage

[0310] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[0311] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0312] 9. Monetization

[0313] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[0314] Specific examples

[0315] Processing user reviews

[0316] 1. A user uses a delivery service and, after completing the delivery, posts a review on the mobile app stating, "This delivery was delayed, but the delivery person's service was excellent."

[0317] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[0318] 3. The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text.

[0319] 4. The server inputs this evaluation data and emotion data into the evaluation model to evaluate the delivery driver's performance. For example, an evaluation score of "4.2 / 5" is generated.

[0320] 5. The server notifies the delivery person's device of the evaluation results and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on the next delivery.

[0321] Determining shipping fees

[0322] 1. The terminal displays the evaluation results to the delivery person and provides suggestions for improvement, with an overall rating of 4.0 and high user satisfaction.

[0323] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on AI suggestions and emotional data.

[0324] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and the customer's emotional state, and calculates the optimal shipping fee.

[0325] This specific implementation of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. Furthermore, by incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, the present invention enables flexible pricing and preferential commission rates based on reviews, allowing delivery personnel and transportation companies to maintain higher levels of customer satisfaction. This is expected to lead to the optimization of the entire delivery service.

[0326] The processing flow will be explained below.

[0327] Step 1:

[0328] A user uses a delivery service and writes a review after delivery is completed.

[0329] Users use a mobile app or web interface to post reviews and ratings about deliveries.

[0330] Step 2:

[0331] The server collects user reviews.

[0332] The server stores the reviews in a database in real time.

[0333] Step 3:

[0334] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0335] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[0336] Step 4:

[0337] The server preprocesses the collected data.

[0338] The server normalizes the review text data, removes invalid data, and imputes missing values.

[0339] Step 5:

[0340] The server uses an emotion engine to recognize emotions from user reviews.

[0341] The server inputs the review text data into an emotion engine, which recognizes and classifies emotions such as "satisfied," "dissatisfied," and "surprise."

[0342] Step 6:

[0343] The server scores the output of the emotion engine.

[0344] The server converts the output of the emotion engine into numerical data and formats it in a way that can be used in the evaluation model.

[0345] Step 7:

[0346] The server combines the emotion data and preprocessed data to generate a dataset for machine learning.

[0347] The server combines the pre-processed data with the emotion data from the emotion engine to create a dataset for machine learning.

[0348] Step 8:

[0349] The server trains the evaluation model using generative AI.

[0350] The server uses machine learning algorithms to generate a delivery evaluation model, with the training dataset including sentiment data.

[0351] Step 9:

[0352] The server inputs the new delivery data into the rating model.

[0353] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[0354] Step 10:

[0355] The server generates the evaluation results.

[0356] The server generates evaluation results regarding delivery quality and efficiency based on the evaluation scores output from the model, which also include feedback based on the user's emotions.

[0357] Step 11:

[0358] The server notifies the delivery person and the transportation company of the evaluation results.

[0359] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[0360] Step 12:

[0361] The terminal displays the evaluation results.

[0362] The terminal (delivery worker's smartphone or tablet) displays the user's emotional feedback along with the evaluation score. This screen also includes suggestions for improvement.

[0363] Step 13:

[0364] The terminal provides the delivery person with the option to set delivery fees.

[0365] The terminal displays a delivery fee setting screen to the delivery person, allowing them to set fees based on evaluation results and emotional data.

[0366] Step 14:

[0367] The server automatically determines the shipping limit.

[0368] The server automatically determines an appropriate upper limit for delivery charges, taking into account delivery conditions, past evaluations of delivery personnel, and recognized user emotional data.

[0369] Step 15:

[0370] The delivery person sets the shipping fee.

[0371] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[0372] Step 16:

[0373] The server manages the registration of sole proprietors.

[0374] The server accepts the registration information of the sole proprietor and verifies the qualifications based on the evaluation criteria.

[0375] Step 17:

[0376] The server calculates and collects fees for using the system.

[0377] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[0378] Example 2

[0379] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Current delivery rating systems only collect user reviews but do not fully utilize their sentiment and specific feedback. Furthermore, there are insufficient means to directly provide feedback on the ratings to delivery personnel and shipping companies, slowing down improvements in delivery quality and efficiency. The current system also has limited functionality for setting delivery fees and preferential handling fees, and the management of setting delivery fee caps and verifying the qualifications of individual business owners is complex. It is necessary to resolve these issues and effectively utilize user reviews and sentiment data to improve overall delivery quality and customer satisfaction.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0382] In this invention, the server includes means for collecting reviews from users, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for providing feedback on the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from text data of reviews and using the emotion data as part of the evaluation, and means for evaluating user satisfaction based on the emotion data and providing the result to delivery personnel. This makes it possible to effectively utilize user emotions and specific feedback to improve overall delivery quality and customer satisfaction.

[0383] "Means for collecting reviews from users" refers to functionality for collecting reviews provided by users via a mobile app or web interface.

[0384] The "means for preprocessing collected reviews and inputting them into the evaluation model" is a function for analyzing collected reviews, converting them into a format suitable for evaluation, and inputting them into the evaluation model.

[0385] "Means for generating a delivery evaluation model using generative artificial intelligence" is a function that uses generative AI technology to build a model for evaluating the quality and efficiency of delivery.

[0386] The "means for evaluating new delivery data using an evaluation model and outputting the evaluation results" is a function that inputs newly collected delivery data into the aforementioned evaluation model and outputs the results as numerical or qualitative evaluations.

[0387] "Means of providing feedback on evaluation results and improving the performance of delivery personnel and transportation companies" refers to a function that notifies the evaluation results to delivery personnel and transportation companies and provides suggestions and actions to improve performance based on the results.

[0388] "Means for recognizing emotions from review text data and using the emotional data as part of the evaluation" is a function that analyzes the review text to recognize the user's emotions and uses the emotional data as part of the evaluation criteria.

[0389] The "means for evaluating user satisfaction based on emotional data and providing the results to the delivery person" is a function that quantifies the user's satisfaction based on the emotionally recognized data and provides the results as feedback to the delivery person.

[0390] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. This system is composed of a user, a terminal, and a server. How each element works together to realize the present invention is described in detail below.

[0391] Data collection

[0392] After using a delivery service, users can enter a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user might post a review stating, "This delivery was delayed, but the delivery person's response was excellent."

[0393] The server collects these user reviews in real time and stores them in a database. It also collects information from the terminals of delivery companies and delivery personnel, and integrates data on property damage and delays into the database.

[0394] Data Preprocessing

[0395] The server preprocesses the collected data, specifically removing inaccurate data, filling in missing values, and analyzing the review text data. This analysis uses natural language processing (NLP) technology to extract keywords such as "delay" and "good service."

[0396] emotion recognition

[0397] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[0398] Generating an evaluation model

[0399] The server creates a dataset for machine learning based on the preprocessed data and sentiment data. Generative AI is used to generate a delivery evaluation model. This evaluation model has criteria for evaluating delivery quality and efficiency. For example, evaluation criteria include timely delivery and flexibility in customer service.

[0400] Conducting the evaluation

[0401] The server evaluates new delivery data using the generated evaluation model. The evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative evaluation (e.g., "very good"). These evaluation results are used as the basis for strategic decision-making and improvement measures for delivery companies.

[0402] feedback

[0403] The server then notifies the delivery person or the transportation company of the evaluation results. The notification includes a breakdown of the evaluation and suggestions for improvement, and provides feedback such as "Overall rating: 4.0" and "User satisfaction: High."

[0404] Emotional Data Feedback

[0405] The server also provides the delivery person with feedback on the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[0406] Expanding system usage

[0407] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[0408] Monetization

[0409] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from delivery companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[0410] Specific examples

[0411] Processing user reviews

[0412] A user submits a review on a mobile app stating, "Delivery was delayed this time, but the delivery person's service was excellent."

[0413] Determining shipping fees

[0414] The delivery person checks the evaluation results on the terminal and attempts to set a new delivery fee.

[0415] This specific embodiment of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. By incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, delivery personnel and transportation companies can maintain higher levels of customer satisfaction.

[0416] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0417] Step 1: Collect reviews

[0418] After using the delivery service, a user posts a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user may post a review stating, "This delivery was delayed, but the delivery person's response was very good."

[0419] Input: User review after delivery service completed

[0420] Output: Reviews stored in a database

[0421] Specific behavior:

[0422] The user logs into the mobile app and opens an interface to enter a review for the specified delivery.

[0423] Write your review and hit submit.

[0424] The server receives the submitted reviews in real time and stores them in a database.

[0425] Step 2: Preprocessing the data

[0426] The server preprocesses the collected reviews. Specifically, it removes inaccurate data, fills in missing values, and analyzes the review text data. Natural language processing (NLP) technology is used for this analysis to extract keywords such as "delay" and "good service."

[0427] Input: Reviews stored in the database

[0428] Output: Preprocessed review text data

[0429] Specific behavior:

[0430] The server retrieves the collected reviews from the database.

[0431] Applying filtering algorithms to remove fraudulent data and spam.

[0432] Apply methods to impute missing data (e.g., mean imputation).

[0433] The review text is analyzed using natural language processing algorithms to extract important keywords and phrases.

[0434] Step 3: Emotion Recognition

[0435] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[0436] Input: Preprocessed text data of reviews

[0437] Output: Sentiment data categorized and scored

[0438] Specific behavior:

[0439] The server runs the sentiment engine (e.g., sentiment analysis algorithm).

[0440] Using text data as input, the system recognizes the user's emotions (e.g., satisfaction, sense of victimization).

[0441] Converting sentiment data into a numerical score, making this data in a format that can be used in rating models.

[0442] Step 4: Generate the evaluation model

[0443] The server creates a machine learning dataset based on the preprocessed data and sentiment data. Generative AI is then used to generate a delivery evaluation model. This evaluation model contains criteria for evaluating delivery quality and efficiency.

[0444] Input: Preprocessed review text and sentiment data

[0445] Output: Shipping evaluation model

[0446] Specific behavior:

[0447] The server builds a dataset to run machine learning algorithms.

[0448] The dataset includes pre-processed data and scored sentiment data.

[0449] Generative AI is used to generate models from this data to evaluate delivery quality and efficiency.

[0450] Train and validate the models and save the best evaluation model.

[0451] Step 5: Conduct the assessment

[0452] The server evaluates new delivery data using the generated evaluation model, and the evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative rating (e.g., "Very Good").

[0453] Input: New delivery data

[0454] Output: Evaluation results (numerical and qualitative evaluation)

[0455] Specific behavior:

[0456] The server inputs the new delivery data into the rating model.

[0457] Run the model and calculate a rating for each element of the shipping data.

[0458] The evaluation results are output in numerical and qualitative report formats.

[0459] Step 6: Feedback of the evaluation results

[0460] The server notifies the delivery person or the transportation company of the evaluation results, including a breakdown of the evaluation and suggestions for improvement.

[0461] Input: Evaluation result

[0462] Output: Feedback (evaluation results, areas for improvement)

[0463] Specific behavior:

[0464] The server creates a feedback message based on the evaluation results.

[0465] A notification is sent to the delivery person or transportation company's device.

[0466] The terminal receives the notification and performs the function of displaying it to the delivery person.

[0467] Step 7: Feedback of emotional data

[0468] The server also feeds back to the delivery person the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[0469] Input: Emotion data

[0470] Output: Feedback content (emotion data)

[0471] Specific behavior:

[0472] The server retrieves the recognized emotion data and formats it for feedback.

[0473] Feedback including emotional data is sent to the delivery person's device.

[0474] The terminal receives the notification and performs the function of displaying it to the delivery person.

[0475] Step 8: Expanding system usage

[0476] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[0477] Input: Evaluation result

[0478] Output: Shipping fee setting options, registration management results

[0479] Specific behavior:

[0480] The terminal displays the evaluation results and presents delivery fee setting options to the delivery person.

[0481] The delivery person will set the shipping fee based on the options provided.

[0482] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0483] Step 9: Monetization

[0484] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[0485] Input: Shipping fee data

[0486] Output: System usage fee calculation result

[0487] Specific behavior:

[0488] The server takes delivery data as input and executes an algorithm to calculate system usage fees.

[0489] We operate a system that automatically collects calculated fees from transportation companies and individual business owners.

[0490] Store the results of fee collection and perform the necessary processes to maintain and operate the system.

[0491] (Application example 2)

[0492] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0493] Conventional delivery evaluation systems are unable to take user emotions into account, making it difficult to provide fast and accurate evaluations and feedback. Furthermore, they lack a mechanism for comprehensively evaluating delivery personnel's performance and customer satisfaction and providing feedback, preventing improvements in delivery quality. Therefore, there is a need for an evaluation system that includes emotion recognition to improve the overall quality of delivery services.

[0494] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0495] In this invention, the server includes means for collecting user reviews, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for feeding back the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from user reviews and using them as part of the rating, and means for providing feedback based on the rating results and emotion data. This enables comprehensive delivery ratings that take user emotions into account, and enables delivery quality to be improved through fast and accurate feedback.

[0496] "User" refers to an individual who uses the Delivery Service and submits a review.

[0497] "Reviews" refer to opinions and impressions posted by users about delivery services.

[0498] "Preprocessing" refers to the process of converting collected reviews into a format suitable for the rating model.

[0499] "Evaluation model" refers to an algorithm generated using generative artificial intelligence to calculate delivery quality and efficiency.

[0500] "Feedback" refers to notifying delivery personnel and transportation companies of the evaluation results and informing them of areas for improvement and the evaluation results.

[0501] "Delivery person" refers to a person who provides delivery services to users.

[0502] "Transportation Company" refers to a company that provides delivery services.

[0503] "Emotion recognition" refers to the technology of analyzing emotions (satisfaction, dissatisfaction, surprise, etc.) from user reviews.

[0504] "Emotion data" refers to data that indicates the user's emotions extracted by emotion recognition.

[0505] "Delivery fee" refers to the fee paid by a user when using a delivery service.

[0506] "Shipping Fee Limit" refers to the maximum shipping fee set based on the shipping terms.

[0507] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[0508] "Independent Business Owner" refers to an individual who registers with the system to provide delivery services.

[0509] "Evaluation criteria" refers to the standards used to evaluate the performance of delivery personnel and transportation companies.

[0510] "Evaluation results" refer to the results of delivery quality and efficiency calculated by the evaluation model.

[0511] The present invention is a system for improving evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. MODES FOR CARRYING OUT THE INVENTION The following describes embodiments of the present invention.

[0512] System configuration

[0513] The system consists of a user, a terminal, and a server. Users use smartphones or PCs to input reviews after delivery services. The terminals are used by delivery personnel and transportation companies and receive evaluation results and areas for improvement as feedback. The server is the core component that collects data, preprocesses it, generates evaluation models, performs evaluations, and provides feedback.

[0514] Hardware and software used

[0515] Hardware:

[0516] Server: An EC2 instance on Amazon Web Services (AWS)

[0517] User device: Smartphone (Android / iOS)

[0518] software:

[0519] Sentiment Analysis: TextBlob (Python library)

[0520] Data preprocessing: scikit-learn (Python library)

[0521] Machine learning model: Support Vector Machine (SVM)

[0522] Data processing and calculation

[0523] The server collects user-entered reviews in real time and stores them in a database. The collected reviews are then preprocessed to convert the text data into an analyzable format. The preprocessed data is then analyzed by an emotion engine to recognize the user's emotional state. This emotion data and the preprocessed reviews are then used to generate a rating model.

[0524] The evaluation model is generated using generative artificial intelligence (generative AI model), and evaluations are performed based on new delivery data. The evaluation results are output as numerical and qualitative evaluations and notified to the terminal. The evaluation results also include areas for improvement, which delivery personnel and transportation companies can use as a reference to improve performance. In addition, user emotional data is notified as feedback, allowing delivery personnel to take specific measures to improve.

[0525] Specific examples

[0526] User review processing example

[0527] A user uses a delivery service and posts a review from a mobile app saying, "This delivery was delayed, but the delivery person's service was excellent." The server stores this review in a database, analyzes the text, and assigns a score of "delay." The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text. The server then inputs the evaluation data and emotion data into an evaluation model to evaluate the delivery person's performance. For example, an evaluation score of "4.2 / 5" is generated. The server notifies the delivery person's device of the evaluation result and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on their next delivery.

[0528] Example prompt statement

[0529] After delivery, a review is posted saying, "This delivery was a little late, but the delivery person's service was excellent." Recognize the user's sentiment from this review and generate a score to evaluate the delivery person's performance.

[0530] This enables comprehensive delivery evaluation that takes into account the user's emotions, and enables delivery quality to be improved through fast and accurate feedback.

[0531] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0532] Step 1:

[0533] After using the delivery service, the user enters a review using a mobile app on their smartphone. The review is sent to the server in real time. Once the review is completed, it is saved in the server's database and becomes available for subsequent processing.

[0534] Step 2:

[0535] The server preprocesses the collected reviews. Specifically, it removes unnecessary characters and symbols from the review text data and converts it into an analyzable format. At this stage, it also removes invalid data and fills in missing values. The input is the reviews entered by the user, and the output is the preprocessed, clean text data.

[0536] Step 3:

[0537] The server inputs the preprocessed reviews into an emotion engine for sentiment analysis. The emotion engine (e.g., TextBlob) recognizes the user's emotion (e.g., satisfied, dissatisfied, etc.) from the review text data. The emotion analysis results are output as a quantitative emotion score.

[0538] Step 4:

[0539] The server generates a rating model using a generative AI model (e.g., support vector machine, SVM) based on the preprocessed reviews and sentiment data. The input is the preprocessed data and sentiment data, and the output is a rating model for the delivery.

[0540] Step 5:

[0541] The server inputs new delivery data into the evaluation model and performs the evaluation, which evaluates the performance of the new delivery. The input is the new delivery data, and the output is an evaluation score and qualitative feedback.

[0542] Step 6:

[0543] The server notifies the terminal of the delivery person or transportation company of the evaluation results. The evaluation results also include areas for improvement, and the delivery person or transportation company uses this feedback to improve their performance. Specifically, the evaluation results and comments are displayed in the application on the terminal.

[0544] Step 7:

[0545] The server also provides emotional feedback, allowing delivery personnel to understand the user's specific emotional state and take measures to improve their next delivery. The emotional data is displayed on the terminal as comments that can help delivery personnel improve their work.

[0546] Step 8:

[0547] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. Delivery people can set delivery fees through a smartphone app based on their own evaluation. The server also sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and the customer's emotional state, and calculates the optimal delivery fee. This is displayed on the terminal, allowing the delivery person to check and adjust it.

[0548] Step 9:

[0549] The server calculates a system usage fee based on the evaluation results and emotion data, and automatically collects it as a percentage of the delivery fee. Fees from delivery companies and individual business owners are automatically calculated and collected via the server's system. Specifically, invoices are generated and sent on the server.

[0550] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0551] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0552] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0553] [Second embodiment]

[0554] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0555] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0556] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0557] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0558] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0560] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0561] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0562] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0563] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0564] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0565] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0566] The present invention aims to improve delivery quality and efficiency by introducing a delivery evaluation system. Specific embodiments for carrying out the present invention will be described below.

[0567] This system consists of a user, a terminal, and a server. The user uses the delivery service and then rates it. Delivery personnel and transportation companies use the terminal to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates rating models, performs the rating, and provides feedback on the results.

[0568] Program processing overview

[0569] 1. Data Collection

[0570] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[0571] The server collects reviews in real time and stores them in a database.

[0572] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0573] 2. Data Preprocessing

[0574] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[0575] 3. Generating the evaluation model

[0576] The server creates a machine learning dataset based on the preprocessed data.

[0577] The server uses artificial intelligence to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[0578] 4. Conducting the evaluation

[0579] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[0580] 5. Feedback of evaluation results

[0581] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[0582] 6. Expanding system usage

[0583] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[0584] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0585] 7. Monetization

[0586] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[0587] Specific examples

[0588] Processing user reviews

[0589] 1. A user actually uses the delivery service and, after the delivery is completed, posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[0590] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[0591] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[0592] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[0593] Determining shipping fees

[0594] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[0595] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions.

[0596] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and calculates the optimal shipping fee.

[0597] This specific form of the present invention enables systematic evaluation and improvement of delivery quality and efficiency, and also realizes flexible pricing and preferential handling fees based on the evaluation, which is expected to optimize the entire delivery service.

[0598] The processing flow will be explained below.

[0599] Step 1:

[0600] A user uses a delivery service and writes a review after delivery is completed.

[0601] Users post reviews and ratings about deliveries using a mobile app or web interface.

[0602] Step 2:

[0603] The server collects user reviews.

[0604] The server stores the reviews in a database in real time.

[0605] Step 3:

[0606] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0607] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[0608] Step 4:

[0609] The server preprocesses the collected data.

[0610] The server normalizes the review text data, removes invalid data, and imputes missing values.

[0611] Step 5:

[0612] The server generates a dataset for machine learning.

[0613] The server extracts features and labels them based on the preprocessed data.

[0614] Step 6:

[0615] The server trains the evaluation model using generative AI.

[0616] The server uses machine learning algorithms to train a rating model for deliveries.

[0617] Step 7:

[0618] The server inputs the new delivery data into the rating model.

[0619] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[0620] Step 8:

[0621] The server generates the evaluation results.

[0622] The server generates evaluation results and improvement suggestions based on the evaluation scores output from the model.

[0623] Step 9:

[0624] The server notifies the delivery person and the transportation company of the evaluation results.

[0625] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[0626] Step 10:

[0627] The terminal displays the evaluation results.

[0628] The terminal (delivery person's smartphone or tablet) displays the evaluation results and improvement suggestions.

[0629] Step 11:

[0630] The terminal provides the delivery person with the option to set delivery fees.

[0631] The terminal displays a delivery fee setting screen to the delivery person, allowing the delivery person to set the delivery fee.

[0632] Step 12:

[0633] The server automatically determines the shipping limit.

[0634] The server automatically sets an appropriate upper limit on delivery charges, taking into account delivery conditions and the delivery person's past evaluations.

[0635] Step 13:

[0636] The delivery person sets the shipping fee.

[0637] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[0638] Step 14:

[0639] The server manages the registration of sole proprietors.

[0640] The server accepts the registration information of the sole proprietor and checks the qualifications based on the evaluation criteria.

[0641] Step 15:

[0642] The server calculates and collects fees for using the system.

[0643] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[0644] Example 1

[0645] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0646] Current delivery services lack a system that accurately evaluates delivery quality and efficiency and provides appropriate feedback to delivery personnel and transportation companies. As a result, it is difficult to improve service quality and efficiency, and user satisfaction declines. In addition, there is a lack of a system that effectively sets appropriate fees and provides preferential treatment for commissions based on evaluations, making it difficult to improve delivery personnel motivation or ensure consistency in service quality.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0648] In this invention, the server includes a means for collecting reviews from users, a means for preprocessing the collected reviews and converting them into a format suitable for evaluation using a natural language processing tool, and a means for creating a machine learning dataset based on the preprocessed data, which enables accurate evaluation of delivery quality and efficiency and provides appropriate feedback.

[0649] "User" refers to a general consumer who uses goods or services.

[0650] "Review" refers to the evaluation or feedback provided by a user after using a product or service.

[0651] "Preprocessing" refers to the process of preparing collected data in a form that can be analyzed and evaluated.

[0652] "Natural language processing tools" refer to software technology for analyzing text data and extracting necessary information.

[0653] A "machine learning dataset" refers to a collection of data used to train a machine learning model.

[0654] "Generative artificial intelligence" refers to programs that have the ability to perform specific tasks using machine learning and deep learning.

[0655] An "evaluation model" refers to an algorithm or system that analyzes data and produces results based on specific evaluation criteria.

[0656] "Delivery Data" means data containing information regarding the progress and results of a delivery.

[0657] "Evaluation Results" refers to the numerical or qualitative evaluations obtained using an evaluation model.

[0658] "Feedback" refers to the process of informing stakeholders of evaluation results and areas for improvement.

[0659] "Fees" refers to the fees paid for use of the Services.

[0660] "Delivery Charge" means the charge payable for delivery of Goods.

[0661] "Maximum shipping fee" refers to the maximum shipping fee set based on the shipping conditions.

[0662] "Sole proprietor" refers to an individual who operates a business as a self-employed person.

[0663] "Evaluation criteria" refers to the standards and indicators used for evaluation.

[0664] "Qualifications" refers to the abilities and conditions necessary to perform a particular role or task.

[0665] The purpose of this invention is to improve delivery quality and efficiency by introducing a delivery evaluation system. This system consists of three entities: a user, a terminal, and a server.

[0666] 1. Data Collection

[0667] After the delivery is complete, the user can leave a review via a mobile app or web interface. For example, the user can write a comment such as "The delivery was 30 minutes later than expected."

[0668] The server collects these reviews in real time and stores them in a database (e.g., MySQL or PostgreSQL). The server is configured to receive and store data using a REST API. The server also collects information about damage and delays from the terminals of delivery companies and delivery personnel. This process is carried out via API, and the latest information is maintained by periodic data transmission from the terminals.

[0669] 2. Data Preprocessing

[0670] The server preprocesses the collected data. Specifically, it analyzes the collected text data using natural language processing tools (e.g., NLTK or SpaCy). This analysis generates the keywords and scoring necessary for evaluation. The server also removes invalid data and completes missing values. For example, invalid data would be data that has clearly been entered incorrectly or is in a non-standard format, and this is eliminated through automatic screening.

[0671] 3. Generating the evaluation model

[0672] The server creates a machine learning dataset based on the preprocessed data. The dataset includes features such as delivery time, rating points, and text reviews. The server then generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch). This model has the ability to train the dataset and evaluate delivery quality and efficiency. The generated model is stored internally on the server for use during evaluation.

[0673] 4. Conducting the evaluation

[0674] The server inputs new delivery data into the evaluation model and calculates the evaluation results. For example, when new delivery data is input into the model, an evaluation score and qualitative comments are automatically generated. The server saves the evaluation results in a database and simultaneously adds them to a notification queue. This process makes the evaluation results available immediately.

[0675] 5. Feedback of evaluation results

[0676] The server notifies the delivery person or transportation company of the evaluation results. This notification is sent instantly using an API or WebSocket. The device receives the notification and displays the evaluation results and areas for improvement to the delivery person. For example, specific feedback such as "Please be punctual" may be included.

[0677] 6. Expanding system usage

[0678] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. The delivery person adjusts the delivery fee based on the evaluation using the terminal UI. The server manages the registration of individual business owners and checks their eligibility based on the evaluation criteria. For example, a warning message is displayed if the evaluation criteria are not met during new registration.

[0679] 7. Monetization

[0680] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This is done using payment gateways such as credit cards and bank transfers. The collected fee is transferred to the operator's account and used to cover system maintenance costs.

[0681] Specific examples

[0682] Processing user reviews

[0683] 1. After completing delivery, the user posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[0684] 2. The server stores this review in a database, analyzes the text (e.g., SpaCy), and scores it as "delayed."

[0685] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[0686] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[0687] Determining shipping fees

[0688] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[0689] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions. For example, the system may display a message saying, "We recommend setting the fee between 1,000 and 1,200 yen."

[0690] 3. The server sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and calculates the optimal delivery fee. For example, dynamic pricing is possible, such as setting an additional fee for late-night deliveries.

[0691] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0692] Step 1:

[0693] The user writes a review after the delivery is completed.

[0694] Specifically, users use a mobile app or web interface to post comments such as "Delivery was 30 minutes later than expected."

[0695] Input: User-entered text review

[0696] Output: Text review data sent to the server

[0697] Step 2:

[0698] The server receives reviews from users and stores them in a database in real time.

[0699] Specifically, the server receives data via a REST API and stores it in a MySQL or PostgreSQL database, validating the data format during this process.

[0700] Input: User-submitted text review

[0701] Output: Review information stored in a database

[0702] Step 3:

[0703] The server preprocesses the collected data.

[0704] Specifically, we use natural language processing tools (e.g., NLTK and SpaCy) to analyze text data and convert it into a format suitable for evaluation. We also remove invalid data and impute missing values.

[0705] Input: Review information stored in the database

[0706] Output: Preprocessed text data

[0707] Step 4:

[0708] The server creates a machine learning dataset based on the preprocessed data.

[0709] Specifically, collected features such as review information, delivery time, and rating points are incorporated into a dataset, which generates a dataset in a format that can be used for rating model training.

[0710] Input: Preprocessed text data

[0711] Output: Dataset for machine learning

[0712] Step 5:

[0713] The server generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch).

[0714] Specifically, the system uses preprocessed data to train a model and generate an evaluation model capable of evaluating delivery quality and efficiency. The generated model is stored on the server.

[0715] Input: Machine learning dataset

[0716] Output: Evaluation model

[0717] Step 6:

[0718] The server inputs new delivery data into the rating model, calculates and outputs the rating results.

[0719] Specifically, new delivery data (e.g., delivery time, user ratings) is input into the evaluation model, and evaluation scores and improvement suggestions are automatically generated.

[0720] Input: New shipping data

[0721] Output: Evaluation results (score, improvement suggestions)

[0722] Step 7:

[0723] The server notifies the delivery person or transportation company of the evaluation results.

[0724] Specifically, the evaluation results are instantly sent via API and WebSocket and displayed on the device, including not only the evaluation score but also points for improvement.

[0725] Input: Evaluation result

[0726] Output: Evaluation results displayed on the terminal of the delivery person or transportation company

[0727] Step 8:

[0728] The terminal provides a shipping fee setting option, and the delivery person sets the shipping fee.

[0729] Specifically, a pricing screen based on AI suggestions is displayed on the device's UI, and the delivery person inputs the optimal delivery fee. Based on this, the server calculates the appropriate fee and sets an upper limit.

[0730] Input: Shipping fee information set on the device

[0731] Output: Shipping fee settings reflected in the system

[0732] Step 9:

[0733] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0734] Specifically, if the evaluation criteria are not met during new registration, a warning message will be displayed. The server will check the qualifications of the sole proprietor based on the evaluation score and decide whether to approve the registration.

[0735] Input: New registration information

[0736] Output: Credential check results and warning messages (if necessary)

[0737] Step 10:

[0738] The server calculates and collects the system usage fee as a percentage of the shipping fee.

[0739] Specifically, fees are automatically collected from transportation companies and individual business owners through a credit card payment gateway and transferred to the operator's account.

[0740] Input: Shipping fee information, handling fee percentage

[0741] Output: Collected fees and transfer details

[0742] (Application example 1)

[0743] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0744] In conventional food delivery services, evaluation of delivery quality and efficiency is subjective, and areas for improvement are not clearly communicated, making it difficult to improve overall service quality. Furthermore, delivery fees are set based on the delivery person's experience and intuition, which can lead to inappropriate pricing, resulting in reduced revenue and customer satisfaction. Furthermore, evaluation criteria are vague, making it difficult for delivery people and transportation companies to obtain useful feedback to optimize their performance. These issues need to be resolved.

[0745] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0746] In this invention, the server includes a means for collecting user reviews, a means for preprocessing the collected reviews and inputting them into a rating model, and a means for generating a delivery rating model using generative artificial intelligence. This enables objective evaluation of delivery quality and efficiency. The server further includes a means for inputting new delivery data into the rating model and outputting the rating results, a means for notifying the user of the rating results and providing delivery fee setting options, and a means for providing feedback on the rating results and suggesting areas for performance improvement. This allows delivery personnel and transportation companies to objectively evaluate their performance and identify specific areas for improvement based on the feedback. The server also includes a means for automatically determining a maximum delivery fee based on delivery conditions, a means for offering preferential commissions to delivery personnel who receive a certain rating or higher based on the rating results, a means for evaluating delivery personnel based on delivery data from the food delivery service, and a means for providing feedback on specific areas for improvement to delivery personnel who do not meet certain rating standards. This enables appropriate delivery fee settings and fair preferential commissions, which is expected to improve the overall quality of the service.

[0747] "User" refers to a person who uses the food delivery service to order and receive meals.

[0748] "Reviews" refer to opinions and impressions provided by users after delivery is completed.

[0749] "Preprocessing" refers to the process of analyzing collected reviews and converting them into a format suitable for the rating model.

[0750] An "evaluation model" refers to a machine learning model that has criteria for evaluating delivery quality and efficiency.

[0751] "Delivery Data" refers to all data related to delivery (delivery time, delay information, user reviews, etc.).

[0752] "Evaluation results" refer to the numerical values ​​and qualitative indicators output by the evaluation model that indicate the quality and efficiency of delivery.

[0753] "Feedback" refers to the points for improvement and evaluation provided based on the evaluation results.

[0754] "Performance" refers to the efficiency and quality of delivery work performed by delivery personnel and transportation companies.

[0755] "Delivery fee" refers to the fee paid by the user when using the delivery service.

[0756] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[0757] "Food delivery service" refers to a service that allows users to order food and drinks online and have them delivered to a specified location.

[0758] "Generative AI" refers to AI that uses machine learning and data analysis to perform specific tasks.

[0759] The present invention aims to evaluate delivery quality and efficiency in food delivery services and improve the overall quality of the service. Specific embodiments for carrying out the present invention will be described below.

[0760] System Configuration

[0761] The system of the present invention is mainly composed of a server, a terminal (such as a smartphone), and a user.

[0762] server:

[0763] The server is the core component that collects data, preprocesses it, generates evaluation models, performs the evaluation, and provides feedback on the results. Specific software used includes Python, Pandas, and Scikit-learn.

[0764] Device:

[0765] The terminal is a device used by delivery personnel and transportation companies to check evaluation results, set delivery fees, and receive feedback. Specifically, this is realized as a smartphone app.

[0766] User:

[0767] Users are people who use food delivery services and post reviews after delivery is complete.

[0768] System Features

[0769] 1. Data Collection:

[0770] When a user uses a food delivery service and the delivery is completed, the user enters a review through the mobile app. The server collects these reviews in real time and stores them in a database.

[0771] 2. Data Preprocessing:

[0772] The server preprocesses the collected data. It analyzes the review text data and converts it into a format suitable for the rating model. For example, if a review says "30 minutes late," it will be scored as "late."

[0773] 3. Generate the evaluation model:

[0774] The server creates a machine learning dataset based on the preprocessed data, generates a random forest model using Scikit-learn, and builds a delivery evaluation model.

[0775] 4. Conducting the evaluation:

[0776] New delivery data is input into the evaluation model, and the evaluation results are calculated. The evaluation results are output as numerical or qualitative evaluations, and the server stores these results.

[0777] 5. Feedback of evaluation results:

[0778] The server then notifies the delivery person or the transportation company of the evaluation results. For example, the delivery person's device may receive feedback such as, "Next time, try to be more punctual."

[0779] 6. Set up shipping rates:

[0780] Based on the evaluation results, the delivery person sets the delivery fee on their smartphone. The server calculates the upper limit of the delivery fee based on delivery conditions (distance, time zone, etc.) and proposes the optimal delivery fee.

[0781] Specific examples

[0782] For example, if a user posts a review on a mobile app stating that "the delivery was 30 minutes later than scheduled," the server stores this review in a database and uses text analysis to score it as "delayed." This data is then input into a rating model, which rates the delivery person's performance as "3.5 / 5." This rating result is sent to the delivery person's smartphone, and feedback such as "Please be punctual next time" is displayed. The delivery person can also adjust the delivery fee based on this result.

[0783] Example prompt sentence:

[0784] "User review: 30 minutes late. Please analyze the cause of the delivery person's delay and provide a rating."

[0785] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0786] Step 1: User submits review after delivery is complete.

[0787] Input: A review a user leaves on a food delivery app (e.g., "Delivery was 30 minutes later than expected").

[0788] Output: User submitted reviews are sent to the server through the app and stored in a database.

[0789] Specific operation: The user uses the smartphone app to write down their thoughts and opinions after the delivery is completed and presses the submit button. The app then sends the review to the server, which records the review in a database.

[0790] Step 2: The server preprocesses the collected reviews.

[0791] Input: User reviews collected in step 1.

[0792] Output: The text data is converted into a format suitable for the evaluation model (e.g., scored as "delay").

[0793] How it works: The server uses Python and Pandas to parse the review text data, checking for the presence of keywords (e.g., "delay"), and then converts the results into scores in a format suitable for the rating model.

[0794] Step 3: The server generates the evaluation model.

[0795] Input: A preprocessed dataset (e.g., a dataset of "delay" scores).

[0796] Output: A delivery evaluation model (e.g., a random forest model).

[0797] How it works: The server uses Scikit-learn to generate an evaluation model, such as a random forest model, from the preprocessed dataset. The model has criteria for evaluating delivery quality and efficiency.

[0798] Step 4: The server inputs the new delivery data into the rating model and performs the rating.

[0799] Input: New delivery data (e.g., new user reviews and their preprocessing results).

[0800] Output: Evaluation result (e.g., delivery driver performance score "3.5 / 5").

[0801] Specific operation: The server inputs new delivery data into the evaluation model to evaluate the delivery person's performance. The evaluation results are generated as numerical values ​​and stored in the database.

[0802] Step 5: The server notifies the delivery person's terminal of the evaluation result.

[0803] Input: Evaluation result (e.g., performance score "3.5 / 5" with specific feedback).

[0804] Output: A notification is sent to the delivery person's smartphone (e.g., "Please be punctual next time").

[0805] Specific operation: The server compiles the evaluation results and generates a notification message. It then sends the message to the delivery person through a smartphone app. The delivery person receives the notification and checks the feedback on the app.

[0806] Step 6: The delivery person sets the shipping fee based on the evaluation results.

[0807] Input: Evaluation results and AI-generated shipping fee suggestions.

[0808] Output: New shipping price configuration (e.g. suggested optimal shipping price).

[0809] Specific operation: The delivery person opens the delivery fee setting screen on their smartphone app and determines the fee based on the AI's suggestions. The server calculates the maximum delivery fee based on delivery conditions (distance, time zone, etc.) and displays the optimal delivery fee. The delivery person sets the delivery fee according to the suggestions and sends it to the server.

[0810] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0811] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. Below, we will explain in detail how each component works together to realize the present invention.

[0812] This system consists of a user, a terminal, and a server. Users use delivery services and then rate them. Delivery personnel and shipping companies use their terminals to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates a rating model, performs the rating, and provides feedback on the results. Furthermore, an emotion engine is used to recognize emotions from user reviews and use them as part of the rating.

[0813] Program processing overview

[0814] 1. Data Collection

[0815] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[0816] The server collects reviews in real time and stores them in a database.

[0817] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0818] 2. Data Preprocessing

[0819] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[0820] 3. Emotion recognition

[0821] The server uses an emotion engine to recognize the user's emotion (e.g., satisfaction, dissatisfaction, surprise, etc.) from the review text data.

[0822] The emotion engine categorizes and scores the emotional data it recognizes.

[0823] 4. Generating the evaluation model

[0824] The server creates a machine learning dataset based on the preprocessed data and emotion data.

[0825] The server uses generative AI to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[0826] 5. Conducting the evaluation

[0827] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[0828] 6. Feedback of evaluation results

[0829] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[0830] 7. Emotional Data Feedback

[0831] The server provides the delivery person with the user's emotional data recognized by the emotion engine as feedback, allowing the delivery person to understand the user's specific emotional state and take measures to improve the situation.

[0832] 8. Expanding System Usage

[0833] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[0834] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[0835] 9. Monetization

[0836] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[0837] Specific examples

[0838] Processing user reviews

[0839] 1. A user uses a delivery service and, after completing the delivery, posts a review on the mobile app stating, "This delivery was delayed, but the delivery person's service was excellent."

[0840] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[0841] 3. The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text.

[0842] 4. The server inputs this evaluation data and emotion data into the evaluation model to evaluate the delivery driver's performance. For example, an evaluation score of "4.2 / 5" is generated.

[0843] 5. The server notifies the delivery person's device of the evaluation results and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on the next delivery.

[0844] Determining shipping fees

[0845] 1. The terminal displays the evaluation results to the delivery person and provides suggestions for improvement, with an overall rating of 4.0 and high user satisfaction.

[0846] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on AI suggestions and emotional data.

[0847] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and the customer's emotional state, and calculates the optimal shipping fee.

[0848] This specific implementation of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. Furthermore, by incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, the present invention enables flexible pricing and preferential commission rates based on reviews, allowing delivery personnel and transportation companies to maintain higher levels of customer satisfaction. This is expected to lead to the optimization of the entire delivery service.

[0849] The processing flow will be explained below.

[0850] Step 1:

[0851] A user uses a delivery service and writes a review after delivery is completed.

[0852] Users use a mobile app or web interface to post reviews and ratings about deliveries.

[0853] Step 2:

[0854] The server collects user reviews.

[0855] The server stores the reviews in a database in real time.

[0856] Step 3:

[0857] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[0858] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[0859] Step 4:

[0860] The server preprocesses the collected data.

[0861] The server normalizes the review text data, removes invalid data, and imputes missing values.

[0862] Step 5:

[0863] The server uses an emotion engine to recognize emotions from user reviews.

[0864] The server inputs the review text data into an emotion engine, which recognizes and classifies emotions such as "satisfied," "dissatisfied," and "surprise."

[0865] Step 6:

[0866] The server scores the output of the emotion engine.

[0867] The server converts the output of the emotion engine into numerical data and formats it in a way that can be used in the evaluation model.

[0868] Step 7:

[0869] The server combines the emotion data and preprocessed data to generate a dataset for machine learning.

[0870] The server combines the pre-processed data with the emotion data from the emotion engine to create a dataset for machine learning.

[0871] Step 8:

[0872] The server trains the evaluation model using generative AI.

[0873] The server uses machine learning algorithms to generate a delivery evaluation model, with the training dataset including sentiment data.

[0874] Step 9:

[0875] The server inputs the new delivery data into the rating model.

[0876] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[0877] Step 10:

[0878] The server generates the evaluation results.

[0879] The server generates evaluation results regarding delivery quality and efficiency based on the evaluation scores output from the model, which also include feedback based on the user's emotions.

[0880] Step 11:

[0881] The server notifies the delivery person and the transportation company of the evaluation results.

[0882] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[0883] Step 12:

[0884] The terminal displays the evaluation results.

[0885] The terminal (delivery worker's smartphone or tablet) displays the user's emotional feedback along with the evaluation score. This screen also includes suggestions for improvement.

[0886] Step 13:

[0887] The terminal provides the delivery person with the option to set delivery fees.

[0888] The terminal displays a delivery fee setting screen to the delivery person, allowing them to set fees based on evaluation results and emotional data.

[0889] Step 14:

[0890] The server automatically determines the shipping limit.

[0891] The server automatically determines an appropriate upper limit for delivery charges, taking into account delivery conditions, past evaluations of delivery personnel, and recognized user emotional data.

[0892] Step 15:

[0893] The delivery person sets the shipping fee.

[0894] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[0895] Step 16:

[0896] The server manages the registration of sole proprietors.

[0897] The server accepts the registration information of the sole proprietor and verifies the qualifications based on the evaluation criteria.

[0898] Step 17:

[0899] The server calculates and collects fees for using the system.

[0900] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[0901] Example 2

[0902] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] Current delivery rating systems only collect user reviews but do not fully utilize their sentiment and specific feedback. Furthermore, there are insufficient means to directly provide feedback on the ratings to delivery personnel and shipping companies, slowing down improvements in delivery quality and efficiency. The current system also has limited functionality for setting delivery fees and preferential handling fees, and the management of setting delivery fee caps and verifying the qualifications of individual business owners is complex. It is necessary to resolve these issues and effectively utilize user reviews and sentiment data to improve overall delivery quality and customer satisfaction.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0905] In this invention, the server includes means for collecting reviews from users, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for providing feedback on the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from text data of reviews and using the emotion data as part of the evaluation, and means for evaluating user satisfaction based on the emotion data and providing the result to delivery personnel. This makes it possible to effectively utilize user emotions and specific feedback to improve overall delivery quality and customer satisfaction.

[0906] "Means for collecting reviews from users" refers to functionality for collecting reviews provided by users via a mobile app or web interface.

[0907] The "means for preprocessing collected reviews and inputting them into the evaluation model" is a function for analyzing collected reviews, converting them into a format suitable for evaluation, and inputting them into the evaluation model.

[0908] "Means for generating a delivery evaluation model using generative artificial intelligence" is a function that uses generative AI technology to build a model for evaluating the quality and efficiency of delivery.

[0909] The "means for evaluating new delivery data using an evaluation model and outputting the evaluation results" is a function that inputs newly collected delivery data into the aforementioned evaluation model and outputs the results as numerical or qualitative evaluations.

[0910] "Means of providing feedback on evaluation results and improving the performance of delivery personnel and transportation companies" refers to a function that notifies the evaluation results to delivery personnel and transportation companies and provides suggestions and actions to improve performance based on the results.

[0911] "Means for recognizing emotions from review text data and using the emotional data as part of the evaluation" is a function that analyzes the review text to recognize the user's emotions and uses the emotional data as part of the evaluation criteria.

[0912] The "means for evaluating user satisfaction based on emotional data and providing the results to the delivery person" is a function that quantifies the user's satisfaction based on the emotionally recognized data and provides the results as feedback to the delivery person.

[0913] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. This system is composed of a user, a terminal, and a server. How each element works together to realize the present invention is described in detail below.

[0914] Data collection

[0915] After using a delivery service, users can enter a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user might post a review stating, "This delivery was delayed, but the delivery person's response was excellent."

[0916] The server collects these user reviews in real time and stores them in a database. It also collects information from the terminals of delivery companies and delivery personnel, and integrates data on property damage and delays into the database.

[0917] Data Preprocessing

[0918] The server preprocesses the collected data, specifically removing inaccurate data, filling in missing values, and analyzing the review text data. This analysis uses natural language processing (NLP) technology to extract keywords such as "delay" and "good service."

[0919] emotion recognition

[0920] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[0921] Generating an evaluation model

[0922] The server creates a dataset for machine learning based on the preprocessed data and sentiment data. Generative AI is used to generate a delivery evaluation model. This evaluation model has criteria for evaluating delivery quality and efficiency. For example, evaluation criteria include timely delivery and flexibility in customer service.

[0923] Conducting the evaluation

[0924] The server evaluates new delivery data using the generated evaluation model. The evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative evaluation (e.g., "very good"). These evaluation results are used as the basis for strategic decision-making and improvement measures for delivery companies.

[0925] feedback

[0926] The server then notifies the delivery person or the transportation company of the evaluation results. The notification includes a breakdown of the evaluation and suggestions for improvement, and provides feedback such as "Overall rating: 4.0" and "User satisfaction: High."

[0927] Emotional Data Feedback

[0928] The server also provides the delivery person with feedback on the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[0929] Expanding system usage

[0930] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[0931] Monetization

[0932] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from delivery companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[0933] Specific examples

[0934] Processing user reviews

[0935] A user submits a review on a mobile app stating, "Delivery was delayed this time, but the delivery person's service was excellent."

[0936] Determining shipping fees

[0937] The delivery person checks the evaluation results on the terminal and attempts to set a new delivery fee.

[0938] This specific embodiment of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. By incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, delivery personnel and transportation companies can maintain higher levels of customer satisfaction.

[0939] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0940] Step 1: Collect reviews

[0941] After using the delivery service, a user posts a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user may post a review stating, "This delivery was delayed, but the delivery person's response was very good."

[0942] Input: User review after delivery service completed

[0943] Output: Reviews stored in a database

[0944] Specific behavior:

[0945] The user logs into the mobile app and opens an interface to enter a review for the specified delivery.

[0946] Write your review and hit submit.

[0947] The server receives the submitted reviews in real time and stores them in a database.

[0948] Step 2: Preprocessing the data

[0949] The server preprocesses the collected reviews. Specifically, it removes inaccurate data, fills in missing values, and analyzes the review text data. Natural language processing (NLP) technology is used for this analysis to extract keywords such as "delay" and "good service."

[0950] Input: Reviews stored in the database

[0951] Output: Preprocessed review text data

[0952] Specific behavior:

[0953] The server retrieves the collected reviews from the database.

[0954] Applying filtering algorithms to remove fraudulent data and spam.

[0955] Apply methods to impute missing data (e.g., mean imputation).

[0956] The review text is analyzed using natural language processing algorithms to extract important keywords and phrases.

[0957] Step 3: Emotion Recognition

[0958] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[0959] Input: Preprocessed text data of reviews

[0960] Output: Sentiment data categorized and scored

[0961] Specific behavior:

[0962] The server runs the sentiment engine (e.g., sentiment analysis algorithm).

[0963] Using text data as input, the system recognizes the user's emotions (e.g., satisfaction, sense of victimization).

[0964] Converting sentiment data into a numerical score, making this data in a format that can be used in rating models.

[0965] Step 4: Generate the evaluation model

[0966] The server creates a machine learning dataset based on the preprocessed data and sentiment data. Generative AI is then used to generate a delivery evaluation model. This evaluation model contains criteria for evaluating delivery quality and efficiency.

[0967] Input: Preprocessed review text and sentiment data

[0968] Output: Shipping evaluation model

[0969] Specific behavior:

[0970] The server builds a dataset to run machine learning algorithms.

[0971] The dataset includes pre-processed data and scored sentiment data.

[0972] Generative AI is used to generate models from this data to evaluate delivery quality and efficiency.

[0973] Train and validate the models and save the best evaluation model.

[0974] Step 5: Conduct the assessment

[0975] The server evaluates new delivery data using the generated evaluation model, and the evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative rating (e.g., "Very Good").

[0976] Input: New delivery data

[0977] Output: Evaluation results (numerical and qualitative evaluation)

[0978] Specific behavior:

[0979] The server inputs the new delivery data into the rating model.

[0980] Run the model and calculate a rating for each element of the shipping data.

[0981] The evaluation results are output in numerical and qualitative report formats.

[0982] Step 6: Feedback of the evaluation results

[0983] The server notifies the delivery person or the transportation company of the evaluation results, including a breakdown of the evaluation and suggestions for improvement.

[0984] Input: Evaluation result

[0985] Output: Feedback (evaluation results, areas for improvement)

[0986] Specific behavior:

[0987] The server creates a feedback message based on the evaluation results.

[0988] A notification is sent to the delivery person or transportation company's device.

[0989] The terminal receives the notification and performs the function of displaying it to the delivery person.

[0990] Step 7: Feedback of emotional data

[0991] The server also feeds back to the delivery person the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[0992] Input: Emotion data

[0993] Output: Feedback content (emotion data)

[0994] Specific behavior:

[0995] The server retrieves the recognized emotion data and formats it for feedback.

[0996] Feedback including emotional data is sent to the delivery person's device.

[0997] The terminal receives the notification and performs the function of displaying it to the delivery person.

[0998] Step 8: Expanding system usage

[0999] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[1000] Input: Evaluation result

[1001] Output: Shipping fee setting options, registration management results

[1002] Specific behavior:

[1003] The terminal displays the evaluation results and presents delivery fee setting options to the delivery person.

[1004] The delivery person will set the shipping fee based on the options provided.

[1005] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1006] Step 9: Monetization

[1007] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[1008] Input: Shipping fee data

[1009] Output: System usage fee calculation result

[1010] Specific behavior:

[1011] The server takes delivery data as input and executes an algorithm to calculate system usage fees.

[1012] We operate a system that automatically collects calculated fees from transportation companies and individual business owners.

[1013] Store the results of fee collection and perform the necessary processes to maintain and operate the system.

[1014] (Application example 2)

[1015] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1016] Conventional delivery evaluation systems are unable to take user emotions into account, making it difficult to provide fast and accurate evaluations and feedback. Furthermore, they lack a mechanism for comprehensively evaluating delivery personnel's performance and customer satisfaction and providing feedback, preventing improvements in delivery quality. Therefore, there is a need for an evaluation system that includes emotion recognition to improve the overall quality of delivery services.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1018] In this invention, the server includes means for collecting user reviews, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for feeding back the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from user reviews and using them as part of the rating, and means for providing feedback based on the rating results and emotion data. This enables comprehensive delivery ratings that take user emotions into account, and enables delivery quality to be improved through fast and accurate feedback.

[1019] "User" refers to an individual who uses the Delivery Service and submits a review.

[1020] "Reviews" refer to opinions and impressions posted by users about delivery services.

[1021] "Preprocessing" refers to the process of converting collected reviews into a format suitable for the rating model.

[1022] "Evaluation model" refers to an algorithm generated using generative artificial intelligence to calculate delivery quality and efficiency.

[1023] "Feedback" refers to notifying delivery personnel and transportation companies of the evaluation results and informing them of areas for improvement and the evaluation results.

[1024] "Delivery person" refers to a person who provides delivery services to users.

[1025] "Transportation Company" refers to a company that provides delivery services.

[1026] "Emotion recognition" refers to the technology of analyzing emotions (satisfaction, dissatisfaction, surprise, etc.) from user reviews.

[1027] "Emotion data" refers to data that indicates the user's emotions extracted by emotion recognition.

[1028] "Delivery fee" refers to the fee paid by a user when using a delivery service.

[1029] "Shipping Fee Limit" refers to the maximum shipping fee set based on the shipping terms.

[1030] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[1031] "Independent Business Owner" refers to an individual who registers with the system to provide delivery services.

[1032] "Evaluation criteria" refers to the standards used to evaluate the performance of delivery personnel and transportation companies.

[1033] "Evaluation results" refer to the results of delivery quality and efficiency calculated by the evaluation model.

[1034] The present invention is a system for improving evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. MODES FOR CARRYING OUT THE INVENTION The following describes embodiments of the present invention.

[1035] System configuration

[1036] The system consists of a user, a terminal, and a server. Users use smartphones or PCs to input reviews after delivery services. The terminals are used by delivery personnel and transportation companies and receive evaluation results and areas for improvement as feedback. The server is the core component that collects data, preprocesses it, generates evaluation models, performs evaluations, and provides feedback.

[1037] Hardware and software used

[1038] Hardware:

[1039] Server: An EC2 instance on Amazon Web Services (AWS)

[1040] User device: Smartphone (Android / iOS)

[1041] software:

[1042] Sentiment Analysis: TextBlob (Python library)

[1043] Data preprocessing: scikit-learn (Python library)

[1044] Machine learning model: Support Vector Machine (SVM)

[1045] Data processing and calculation

[1046] The server collects user-entered reviews in real time and stores them in a database. The collected reviews are then preprocessed to convert the text data into an analyzable format. The preprocessed data is then analyzed by an emotion engine to recognize the user's emotional state. This emotion data and the preprocessed reviews are then used to generate a rating model.

[1047] The evaluation model is generated using generative artificial intelligence (generative AI model), and evaluations are performed based on new delivery data. The evaluation results are output as numerical and qualitative evaluations and notified to the terminal. The evaluation results also include areas for improvement, which delivery personnel and transportation companies can use as a reference to improve performance. In addition, user emotional data is notified as feedback, allowing delivery personnel to take specific measures to improve.

[1048] Specific examples

[1049] User review processing example

[1050] A user uses a delivery service and posts a review from a mobile app saying, "This delivery was delayed, but the delivery person's service was excellent." The server stores this review in a database, analyzes the text, and assigns a score of "delay." The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text. The server then inputs the evaluation data and emotion data into an evaluation model to evaluate the delivery person's performance. For example, an evaluation score of "4.2 / 5" is generated. The server notifies the delivery person's device of the evaluation result and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on their next delivery.

[1051] Example prompt statement

[1052] After delivery, a review is posted saying, "This delivery was a little late, but the delivery person's service was excellent." Recognize the user's sentiment from this review and generate a score to evaluate the delivery person's performance.

[1053] This enables comprehensive delivery evaluation that takes into account the user's emotions, and enables delivery quality to be improved through fast and accurate feedback.

[1054] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1055] Step 1:

[1056] After using the delivery service, the user enters a review using a mobile app on their smartphone. The review is sent to the server in real time. Once the review is completed, it is saved in the server's database and becomes available for subsequent processing.

[1057] Step 2:

[1058] The server preprocesses the collected reviews. Specifically, it removes unnecessary characters and symbols from the review text data and converts it into an analyzable format. At this stage, it also removes invalid data and fills in missing values. The input is the reviews entered by the user, and the output is the preprocessed, clean text data.

[1059] Step 3:

[1060] The server inputs the preprocessed reviews into an emotion engine for sentiment analysis. The emotion engine (e.g., TextBlob) recognizes the user's emotion (e.g., satisfied, dissatisfied, etc.) from the review text data. The emotion analysis results are output as a quantitative emotion score.

[1061] Step 4:

[1062] The server generates a rating model using a generative AI model (e.g., support vector machine, SVM) based on the preprocessed reviews and sentiment data. The input is the preprocessed data and sentiment data, and the output is a rating model for the delivery.

[1063] Step 5:

[1064] The server inputs new delivery data into the evaluation model and performs the evaluation, which evaluates the performance of the new delivery. The input is the new delivery data, and the output is an evaluation score and qualitative feedback.

[1065] Step 6:

[1066] The server notifies the terminal of the delivery person or transportation company of the evaluation results. The evaluation results also include areas for improvement, and the delivery person or transportation company uses this feedback to improve their performance. Specifically, the evaluation results and comments are displayed in the application on the terminal.

[1067] Step 7:

[1068] The server also provides emotional feedback, allowing delivery personnel to understand the user's specific emotional state and take measures to improve their next delivery. The emotional data is displayed on the terminal as comments that can help delivery personnel improve their work.

[1069] Step 8:

[1070] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. Delivery people can set delivery fees through a smartphone app based on their own evaluation. The server also sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and the customer's emotional state, and calculates the optimal delivery fee. This is displayed on the terminal, allowing the delivery person to check and adjust it.

[1071] Step 9:

[1072] The server calculates a system usage fee based on the evaluation results and emotion data, and automatically collects it as a percentage of the delivery fee. Fees from delivery companies and individual business owners are automatically calculated and collected via the server's system. Specifically, invoices are generated and sent on the server.

[1073] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1074] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1075] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1076] [Third embodiment]

[1077] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1078] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1079] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1080] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1081] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1082] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1083] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1084] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1085] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1086] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1087] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1088] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1089] The present invention aims to improve delivery quality and efficiency by introducing a delivery evaluation system. Specific embodiments for carrying out the present invention will be described below.

[1090] This system consists of a user, a terminal, and a server. The user uses the delivery service and then rates it. Delivery personnel and transportation companies use the terminal to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates rating models, performs the rating, and provides feedback on the results.

[1091] Program processing overview

[1092] 1. Data Collection

[1093] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[1094] The server collects reviews in real time and stores them in a database.

[1095] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1096] 2. Data Preprocessing

[1097] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[1098] 3. Generating the evaluation model

[1099] The server creates a machine learning dataset based on the preprocessed data.

[1100] The server uses artificial intelligence to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[1101] 4. Conducting the evaluation

[1102] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[1103] 5. Feedback of evaluation results

[1104] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[1105] 6. Expanding system usage

[1106] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[1107] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1108] 7. Monetization

[1109] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[1110] Specific examples

[1111] Processing user reviews

[1112] 1. A user actually uses the delivery service and, after the delivery is completed, posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[1113] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[1114] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[1115] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[1116] Determining shipping fees

[1117] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[1118] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions.

[1119] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and calculates the optimal shipping fee.

[1120] This specific form of the present invention enables systematic evaluation and improvement of delivery quality and efficiency, and also realizes flexible pricing and preferential handling fees based on the evaluation, which is expected to optimize the entire delivery service.

[1121] The processing flow will be explained below.

[1122] Step 1:

[1123] A user uses a delivery service and writes a review after delivery is completed.

[1124] Users post reviews and ratings about deliveries using a mobile app or web interface.

[1125] Step 2:

[1126] The server collects user reviews.

[1127] The server stores the reviews in a database in real time.

[1128] Step 3:

[1129] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1130] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[1131] Step 4:

[1132] The server preprocesses the collected data.

[1133] The server normalizes the review text data, removes invalid data, and imputes missing values.

[1134] Step 5:

[1135] The server generates a dataset for machine learning.

[1136] The server extracts features and labels them based on the preprocessed data.

[1137] Step 6:

[1138] The server trains the evaluation model using generative AI.

[1139] The server uses machine learning algorithms to train a rating model for deliveries.

[1140] Step 7:

[1141] The server inputs the new delivery data into the rating model.

[1142] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[1143] Step 8:

[1144] The server generates the evaluation results.

[1145] The server generates evaluation results and improvement suggestions based on the evaluation scores output from the model.

[1146] Step 9:

[1147] The server notifies the delivery person and the transportation company of the evaluation results.

[1148] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[1149] Step 10:

[1150] The terminal displays the evaluation results.

[1151] The terminal (delivery person's smartphone or tablet) displays the evaluation results and improvement suggestions.

[1152] Step 11:

[1153] The terminal provides the delivery person with the option to set delivery fees.

[1154] The terminal displays a delivery fee setting screen to the delivery person, allowing the delivery person to set the delivery fee.

[1155] Step 12:

[1156] The server automatically determines the shipping limit.

[1157] The server automatically sets an appropriate upper limit on delivery charges, taking into account delivery conditions and the delivery person's past evaluations.

[1158] Step 13:

[1159] The delivery person sets the shipping fee.

[1160] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[1161] Step 14:

[1162] The server manages the registration of sole proprietors.

[1163] The server accepts the registration information of the sole proprietor and checks the qualifications based on the evaluation criteria.

[1164] Step 15:

[1165] The server calculates and collects fees for using the system.

[1166] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] Current delivery services lack a system that accurately evaluates delivery quality and efficiency and provides appropriate feedback to delivery personnel and transportation companies. As a result, it is difficult to improve service quality and efficiency, and user satisfaction declines. In addition, there is a lack of a system that effectively sets appropriate fees and provides preferential treatment for commissions based on evaluations, making it difficult to improve delivery personnel motivation or ensure consistency in service quality.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes a means for collecting reviews from users, a means for preprocessing the collected reviews and converting them into a format suitable for evaluation using a natural language processing tool, and a means for creating a machine learning dataset based on the preprocessed data, which enables accurate evaluation of delivery quality and efficiency and provides appropriate feedback.

[1172] "User" refers to a general consumer who uses goods or services.

[1173] "Review" refers to the evaluation or feedback provided by a user after using a product or service.

[1174] "Preprocessing" refers to the process of preparing collected data in a form that can be analyzed and evaluated.

[1175] "Natural language processing tools" refer to software technology for analyzing text data and extracting necessary information.

[1176] A "machine learning dataset" refers to a collection of data used to train a machine learning model.

[1177] "Generative artificial intelligence" refers to programs that have the ability to perform specific tasks using machine learning and deep learning.

[1178] An "evaluation model" refers to an algorithm or system that analyzes data and produces results based on specific evaluation criteria.

[1179] "Delivery Data" means data containing information regarding the progress and results of a delivery.

[1180] "Evaluation Results" refers to the numerical or qualitative evaluations obtained using an evaluation model.

[1181] "Feedback" refers to the process of informing stakeholders of evaluation results and areas for improvement.

[1182] "Fees" refers to the fees paid for use of the Services.

[1183] "Delivery Charge" means the charge payable for delivery of Goods.

[1184] "Maximum shipping fee" refers to the maximum shipping fee set based on the shipping conditions.

[1185] "Sole proprietor" refers to an individual who operates a business as a self-employed person.

[1186] "Evaluation criteria" refers to the standards and indicators used for evaluation.

[1187] "Qualifications" refers to the abilities and conditions necessary to perform a particular role or task.

[1188] The purpose of this invention is to improve delivery quality and efficiency by introducing a delivery evaluation system. This system consists of three entities: a user, a terminal, and a server.

[1189] 1. Data Collection

[1190] After the delivery is complete, the user can leave a review via a mobile app or web interface. For example, the user can write a comment such as "The delivery was 30 minutes later than expected."

[1191] The server collects these reviews in real time and stores them in a database (e.g., MySQL or PostgreSQL). The server is configured to receive and store data using a REST API. The server also collects information about damage and delays from the terminals of delivery companies and delivery personnel. This process is carried out via API, and the latest information is maintained by periodic data transmission from the terminals.

[1192] 2. Data Preprocessing

[1193] The server preprocesses the collected data. Specifically, it analyzes the collected text data using natural language processing tools (e.g., NLTK or SpaCy). This analysis generates the keywords and scoring necessary for evaluation. The server also removes invalid data and completes missing values. For example, invalid data would be data that has clearly been entered incorrectly or is in a non-standard format, and this is eliminated through automatic screening.

[1194] 3. Generating the evaluation model

[1195] The server creates a machine learning dataset based on the preprocessed data. The dataset includes features such as delivery time, rating points, and text reviews. The server then generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch). This model has the ability to train the dataset and evaluate delivery quality and efficiency. The generated model is stored internally on the server for use during evaluation.

[1196] 4. Conducting the evaluation

[1197] The server inputs new delivery data into the evaluation model and calculates the evaluation results. For example, when new delivery data is input into the model, an evaluation score and qualitative comments are automatically generated. The server saves the evaluation results in a database and simultaneously adds them to a notification queue. This process makes the evaluation results available immediately.

[1198] 5. Feedback of evaluation results

[1199] The server notifies the delivery person or transportation company of the evaluation results. This notification is sent instantly using an API or WebSocket. The device receives the notification and displays the evaluation results and areas for improvement to the delivery person. For example, specific feedback such as "Please be punctual" may be included.

[1200] 6. Expanding system usage

[1201] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. The delivery person adjusts the delivery fee based on the evaluation using the terminal UI. The server manages the registration of individual business owners and checks their eligibility based on the evaluation criteria. For example, a warning message is displayed if the evaluation criteria are not met during new registration.

[1202] 7. Monetization

[1203] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This is done using payment gateways such as credit cards and bank transfers. The collected fee is transferred to the operator's account and used to cover system maintenance costs.

[1204] Specific examples

[1205] Processing user reviews

[1206] 1. After completing delivery, the user posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[1207] 2. The server stores this review in a database, analyzes the text (e.g., SpaCy), and scores it as "delayed."

[1208] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[1209] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[1210] Determining shipping fees

[1211] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[1212] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions. For example, the system may display a message saying, "We recommend setting the fee between 1,000 and 1,200 yen."

[1213] 3. The server sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and calculates the optimal delivery fee. For example, dynamic pricing is possible, such as setting an additional fee for late-night deliveries.

[1214] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1215] Step 1:

[1216] The user writes a review after the delivery is completed.

[1217] Specifically, users use a mobile app or web interface to post comments such as "Delivery was 30 minutes later than expected."

[1218] Input: User-entered text review

[1219] Output: Text review data sent to the server

[1220] Step 2:

[1221] The server receives reviews from users and stores them in a database in real time.

[1222] Specifically, the server receives data via a REST API and stores it in a MySQL or PostgreSQL database, validating the data format during this process.

[1223] Input: User-submitted text review

[1224] Output: Review information stored in a database

[1225] Step 3:

[1226] The server preprocesses the collected data.

[1227] Specifically, we use natural language processing tools (e.g., NLTK and SpaCy) to analyze text data and convert it into a format suitable for evaluation. We also remove invalid data and impute missing values.

[1228] Input: Review information stored in the database

[1229] Output: Preprocessed text data

[1230] Step 4:

[1231] The server creates a machine learning dataset based on the preprocessed data.

[1232] Specifically, collected features such as review information, delivery time, and rating points are incorporated into a dataset, which generates a dataset in a format that can be used for rating model training.

[1233] Input: Preprocessed text data

[1234] Output: Dataset for machine learning

[1235] Step 5:

[1236] The server generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch).

[1237] Specifically, the system uses preprocessed data to train a model and generate an evaluation model capable of evaluating delivery quality and efficiency. The generated model is stored on the server.

[1238] Input: Machine learning dataset

[1239] Output: Evaluation model

[1240] Step 6:

[1241] The server inputs new delivery data into the rating model, calculates and outputs the rating results.

[1242] Specifically, new delivery data (e.g., delivery time, user ratings) is input into the evaluation model, and evaluation scores and improvement suggestions are automatically generated.

[1243] Input: New shipping data

[1244] Output: Evaluation results (score, improvement suggestions)

[1245] Step 7:

[1246] The server notifies the delivery person or transportation company of the evaluation results.

[1247] Specifically, the evaluation results are instantly sent via API and WebSocket and displayed on the device, including not only the evaluation score but also points for improvement.

[1248] Input: Evaluation result

[1249] Output: Evaluation results displayed on the terminal of the delivery person or transportation company

[1250] Step 8:

[1251] The terminal provides a shipping fee setting option, and the delivery person sets the shipping fee.

[1252] Specifically, a pricing screen based on AI suggestions is displayed on the device's UI, and the delivery person inputs the optimal delivery fee. Based on this, the server calculates the appropriate fee and sets an upper limit.

[1253] Input: Shipping fee information set on the device

[1254] Output: Shipping fee settings reflected in the system

[1255] Step 9:

[1256] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1257] Specifically, if the evaluation criteria are not met during new registration, a warning message will be displayed. The server will check the qualifications of the sole proprietor based on the evaluation score and decide whether to approve the registration.

[1258] Input: New registration information

[1259] Output: Credential check results and warning messages (if necessary)

[1260] Step 10:

[1261] The server calculates and collects the system usage fee as a percentage of the shipping fee.

[1262] Specifically, fees are automatically collected from transportation companies and individual business owners through a credit card payment gateway and transferred to the operator's account.

[1263] Input: Shipping fee information, handling fee percentage

[1264] Output: Collected fees and transfer details

[1265] (Application example 1)

[1266] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1267] In conventional food delivery services, evaluation of delivery quality and efficiency is subjective, and areas for improvement are not clearly communicated, making it difficult to improve overall service quality. Furthermore, delivery fees are set based on the delivery person's experience and intuition, which can lead to inappropriate pricing, resulting in reduced revenue and customer satisfaction. Furthermore, evaluation criteria are vague, making it difficult for delivery people and transportation companies to obtain useful feedback to optimize their performance. These issues need to be resolved.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1269] In this invention, the server includes a means for collecting user reviews, a means for preprocessing the collected reviews and inputting them into a rating model, and a means for generating a delivery rating model using generative artificial intelligence. This enables objective evaluation of delivery quality and efficiency. The server further includes a means for inputting new delivery data into the rating model and outputting the rating results, a means for notifying the user of the rating results and providing delivery fee setting options, and a means for providing feedback on the rating results and suggesting areas for performance improvement. This allows delivery personnel and transportation companies to objectively evaluate their performance and identify specific areas for improvement based on the feedback. The server also includes a means for automatically determining a maximum delivery fee based on delivery conditions, a means for offering preferential commissions to delivery personnel who receive a certain rating or higher based on the rating results, a means for evaluating delivery personnel based on delivery data from the food delivery service, and a means for providing feedback on specific areas for improvement to delivery personnel who do not meet certain rating standards. This enables appropriate delivery fee settings and fair preferential commissions, which is expected to improve the overall quality of the service.

[1270] "User" refers to a person who uses the food delivery service to order and receive meals.

[1271] "Reviews" refer to opinions and impressions provided by users after delivery is completed.

[1272] "Preprocessing" refers to the process of analyzing collected reviews and converting them into a format suitable for the rating model.

[1273] An "evaluation model" refers to a machine learning model that has criteria for evaluating delivery quality and efficiency.

[1274] "Delivery Data" refers to all data related to delivery (delivery time, delay information, user reviews, etc.).

[1275] "Evaluation results" refer to the numerical values ​​and qualitative indicators output by the evaluation model that indicate the quality and efficiency of delivery.

[1276] "Feedback" refers to the points for improvement and evaluation provided based on the evaluation results.

[1277] "Performance" refers to the efficiency and quality of delivery work performed by delivery personnel and transportation companies.

[1278] "Delivery fee" refers to the fee paid by the user when using the delivery service.

[1279] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[1280] "Food delivery service" refers to a service that allows users to order food and drinks online and have them delivered to a specified location.

[1281] "Generative AI" refers to AI that uses machine learning and data analysis to perform specific tasks.

[1282] The present invention aims to evaluate delivery quality and efficiency in food delivery services and improve the overall quality of the service. Specific embodiments for carrying out the present invention will be described below.

[1283] System Configuration

[1284] The system of the present invention is mainly composed of a server, a terminal (such as a smartphone), and a user.

[1285] server:

[1286] The server is the core component that collects data, preprocesses it, generates evaluation models, performs the evaluation, and provides feedback on the results. Specific software used includes Python, Pandas, and Scikit-learn.

[1287] Device:

[1288] The terminal is a device used by delivery personnel and transportation companies to check evaluation results, set delivery fees, and receive feedback. Specifically, this is realized as a smartphone app.

[1289] User:

[1290] Users are people who use food delivery services and post reviews after delivery is complete.

[1291] System Features

[1292] 1. Data Collection:

[1293] When a user uses a food delivery service and the delivery is completed, the user enters a review through the mobile app. The server collects these reviews in real time and stores them in a database.

[1294] 2. Data Preprocessing:

[1295] The server preprocesses the collected data. It analyzes the review text data and converts it into a format suitable for the rating model. For example, if a review says "30 minutes late," it will be scored as "late."

[1296] 3. Generate the evaluation model:

[1297] The server creates a machine learning dataset based on the preprocessed data, generates a random forest model using Scikit-learn, and builds a delivery evaluation model.

[1298] 4. Conducting the evaluation:

[1299] New delivery data is input into the evaluation model, and the evaluation results are calculated. The evaluation results are output as numerical or qualitative evaluations, and the server stores these results.

[1300] 5. Feedback of evaluation results:

[1301] The server then notifies the delivery person or the transportation company of the evaluation results. For example, the delivery person's device may receive feedback such as, "Next time, try to be more punctual."

[1302] 6. Set up shipping rates:

[1303] Based on the evaluation results, the delivery person sets the delivery fee on their smartphone. The server calculates the upper limit of the delivery fee based on delivery conditions (distance, time zone, etc.) and proposes the optimal delivery fee.

[1304] Specific examples

[1305] For example, if a user posts a review on a mobile app stating that "the delivery was 30 minutes later than scheduled," the server stores this review in a database and uses text analysis to score it as "delayed." This data is then input into a rating model, which rates the delivery person's performance as "3.5 / 5." This rating result is sent to the delivery person's smartphone, and feedback such as "Please be punctual next time" is displayed. The delivery person can also adjust the delivery fee based on this result.

[1306] Example prompt sentence:

[1307] "User review: 30 minutes late. Please analyze the cause of the delivery person's delay and provide a rating."

[1308] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1309] Step 1: User submits review after delivery is complete.

[1310] Input: A review a user leaves on a food delivery app (e.g., "Delivery was 30 minutes later than expected").

[1311] Output: User submitted reviews are sent to the server through the app and stored in a database.

[1312] Specific operation: The user uses the smartphone app to write down their thoughts and opinions after the delivery is completed and presses the submit button. The app then sends the review to the server, which records the review in a database.

[1313] Step 2: The server preprocesses the collected reviews.

[1314] Input: User reviews collected in step 1.

[1315] Output: The text data is converted into a format suitable for the evaluation model (e.g., scored as "delay").

[1316] How it works: The server uses Python and Pandas to parse the review text data, checking for the presence of keywords (e.g., "delay"), and then converts the results into scores in a format suitable for the rating model.

[1317] Step 3: The server generates the evaluation model.

[1318] Input: A preprocessed dataset (e.g., a dataset of "delay" scores).

[1319] Output: A delivery evaluation model (e.g., a random forest model).

[1320] How it works: The server uses Scikit-learn to generate an evaluation model, such as a random forest model, from the preprocessed dataset. The model has criteria for evaluating delivery quality and efficiency.

[1321] Step 4: The server inputs the new delivery data into the rating model and performs the rating.

[1322] Input: New delivery data (e.g., new user reviews and their preprocessing results).

[1323] Output: Evaluation result (e.g., delivery driver performance score "3.5 / 5").

[1324] Specific operation: The server inputs new delivery data into the evaluation model to evaluate the delivery person's performance. The evaluation results are generated as numerical values ​​and stored in the database.

[1325] Step 5: The server notifies the delivery person's terminal of the evaluation result.

[1326] Input: Evaluation result (e.g., performance score "3.5 / 5" with specific feedback).

[1327] Output: A notification is sent to the delivery person's smartphone (e.g., "Please be punctual next time").

[1328] Specific operation: The server compiles the evaluation results and generates a notification message. It then sends the message to the delivery person through a smartphone app. The delivery person receives the notification and checks the feedback on the app.

[1329] Step 6: The delivery person sets the shipping fee based on the evaluation results.

[1330] Input: Evaluation results and AI-generated shipping fee suggestions.

[1331] Output: New shipping price configuration (e.g. suggested optimal shipping price).

[1332] Specific operation: The delivery person opens the delivery fee setting screen on their smartphone app and determines the fee based on the AI's suggestions. The server calculates the maximum delivery fee based on delivery conditions (distance, time zone, etc.) and displays the optimal delivery fee. The delivery person sets the delivery fee according to the suggestions and sends it to the server.

[1333] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1334] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. Below, we will explain in detail how each component works together to realize the present invention.

[1335] This system consists of a user, a terminal, and a server. Users use delivery services and then rate them. Delivery personnel and shipping companies use their terminals to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates a rating model, performs the rating, and provides feedback on the results. Furthermore, an emotion engine is used to recognize emotions from user reviews and use them as part of the rating.

[1336] Program processing overview

[1337] 1. Data Collection

[1338] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[1339] The server collects reviews in real time and stores them in a database.

[1340] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1341] 2. Data Preprocessing

[1342] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[1343] 3. Emotion recognition

[1344] The server uses an emotion engine to recognize the user's emotion (e.g., satisfaction, dissatisfaction, surprise, etc.) from the review text data.

[1345] The emotion engine categorizes and scores the emotional data it recognizes.

[1346] 4. Generating the evaluation model

[1347] The server creates a machine learning dataset based on the preprocessed data and emotion data.

[1348] The server uses generative AI to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[1349] 5. Conducting the evaluation

[1350] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[1351] 6. Feedback of evaluation results

[1352] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[1353] 7. Emotional Data Feedback

[1354] The server provides the delivery person with the user's emotional data recognized by the emotion engine as feedback, allowing the delivery person to understand the user's specific emotional state and take measures to improve the situation.

[1355] 8. Expanding System Usage

[1356] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[1357] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1358] 9. Monetization

[1359] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[1360] Specific examples

[1361] Processing user reviews

[1362] 1. A user uses a delivery service and, after completing the delivery, posts a review on the mobile app stating, "This delivery was delayed, but the delivery person's service was excellent."

[1363] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[1364] 3. The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text.

[1365] 4. The server inputs this evaluation data and emotion data into the evaluation model to evaluate the delivery driver's performance. For example, an evaluation score of "4.2 / 5" is generated.

[1366] 5. The server notifies the delivery person's device of the evaluation results and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on the next delivery.

[1367] Determining shipping fees

[1368] 1. The terminal displays the evaluation results to the delivery person and provides suggestions for improvement, with an overall rating of 4.0 and high user satisfaction.

[1369] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on AI suggestions and emotional data.

[1370] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and the customer's emotional state, and calculates the optimal shipping fee.

[1371] This specific implementation of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. Furthermore, by incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, the present invention enables flexible pricing and preferential commission rates based on reviews, allowing delivery personnel and transportation companies to maintain higher levels of customer satisfaction. This is expected to lead to the optimization of the entire delivery service.

[1372] The processing flow will be explained below.

[1373] Step 1:

[1374] A user uses a delivery service and writes a review after delivery is completed.

[1375] Users use a mobile app or web interface to post reviews and ratings about deliveries.

[1376] Step 2:

[1377] The server collects user reviews.

[1378] The server stores the reviews in a database in real time.

[1379] Step 3:

[1380] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1381] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[1382] Step 4:

[1383] The server preprocesses the collected data.

[1384] The server normalizes the review text data, removes invalid data, and imputes missing values.

[1385] Step 5:

[1386] The server uses an emotion engine to recognize emotions from user reviews.

[1387] The server inputs the review text data into an emotion engine, which recognizes and classifies emotions such as "satisfied," "dissatisfied," and "surprise."

[1388] Step 6:

[1389] The server scores the output of the emotion engine.

[1390] The server converts the output of the emotion engine into numerical data and formats it in a way that can be used in the evaluation model.

[1391] Step 7:

[1392] The server combines the emotion data and preprocessed data to generate a dataset for machine learning.

[1393] The server combines the pre-processed data with the emotion data from the emotion engine to create a dataset for machine learning.

[1394] Step 8:

[1395] The server trains the evaluation model using generative AI.

[1396] The server uses machine learning algorithms to generate a delivery evaluation model, with the training dataset including sentiment data.

[1397] Step 9:

[1398] The server inputs the new delivery data into the rating model.

[1399] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[1400] Step 10:

[1401] The server generates the evaluation results.

[1402] The server generates evaluation results regarding delivery quality and efficiency based on the evaluation scores output from the model, which also include feedback based on the user's emotions.

[1403] Step 11:

[1404] The server notifies the delivery person and the transportation company of the evaluation results.

[1405] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[1406] Step 12:

[1407] The terminal displays the evaluation results.

[1408] The terminal (delivery worker's smartphone or tablet) displays the user's emotional feedback along with the evaluation score. This screen also includes suggestions for improvement.

[1409] Step 13:

[1410] The terminal provides the delivery person with the option to set delivery fees.

[1411] The terminal displays a delivery fee setting screen to the delivery person, allowing them to set fees based on evaluation results and emotional data.

[1412] Step 14:

[1413] The server automatically determines the shipping limit.

[1414] The server automatically determines an appropriate upper limit for delivery charges, taking into account delivery conditions, past evaluations of delivery personnel, and recognized user emotional data.

[1415] Step 15:

[1416] The delivery person sets the shipping fee.

[1417] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[1418] Step 16:

[1419] The server manages the registration of sole proprietors.

[1420] The server accepts the registration information of the sole proprietor and verifies the qualifications based on the evaluation criteria.

[1421] Step 17:

[1422] The server calculates and collects fees for using the system.

[1423] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[1424] Example 2

[1425] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1426] Current delivery rating systems only collect user reviews but do not fully utilize their sentiment and specific feedback. Furthermore, there are insufficient means to directly provide feedback on the ratings to delivery personnel and shipping companies, slowing down improvements in delivery quality and efficiency. The current system also has limited functionality for setting delivery fees and preferential handling fees, and the management of setting delivery fee caps and verifying the qualifications of individual business owners is complex. It is necessary to resolve these issues and effectively utilize user reviews and sentiment data to improve overall delivery quality and customer satisfaction.

[1427] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1428] In this invention, the server includes means for collecting reviews from users, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for providing feedback on the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from text data of reviews and using the emotion data as part of the evaluation, and means for evaluating user satisfaction based on the emotion data and providing the result to delivery personnel. This makes it possible to effectively utilize user emotions and specific feedback to improve overall delivery quality and customer satisfaction.

[1429] "Means for collecting reviews from users" refers to functionality for collecting reviews provided by users via a mobile app or web interface.

[1430] The "means for preprocessing collected reviews and inputting them into the evaluation model" is a function for analyzing collected reviews, converting them into a format suitable for evaluation, and inputting them into the evaluation model.

[1431] "Means for generating a delivery evaluation model using generative artificial intelligence" is a function that uses generative AI technology to build a model for evaluating the quality and efficiency of delivery.

[1432] The "means for evaluating new delivery data using an evaluation model and outputting the evaluation results" is a function that inputs newly collected delivery data into the aforementioned evaluation model and outputs the results as numerical or qualitative evaluations.

[1433] "Means of providing feedback on evaluation results and improving the performance of delivery personnel and transportation companies" refers to a function that notifies the evaluation results to delivery personnel and transportation companies and provides suggestions and actions to improve performance based on the results.

[1434] "Means for recognizing emotions from review text data and using the emotional data as part of the evaluation" is a function that analyzes the review text to recognize the user's emotions and uses the emotional data as part of the evaluation criteria.

[1435] The "means for evaluating user satisfaction based on emotional data and providing the results to the delivery person" is a function that quantifies the user's satisfaction based on the emotionally recognized data and provides the results as feedback to the delivery person.

[1436] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. This system is composed of a user, a terminal, and a server. How each element works together to realize the present invention is described in detail below.

[1437] Data collection

[1438] After using a delivery service, users can enter a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user might post a review stating, "This delivery was delayed, but the delivery person's response was excellent."

[1439] The server collects these user reviews in real time and stores them in a database. It also collects information from the terminals of delivery companies and delivery personnel, and integrates data on property damage and delays into the database.

[1440] Data Preprocessing

[1441] The server preprocesses the collected data, specifically removing inaccurate data, filling in missing values, and analyzing the review text data. This analysis uses natural language processing (NLP) technology to extract keywords such as "delay" and "good service."

[1442] emotion recognition

[1443] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[1444] Generating an evaluation model

[1445] The server creates a dataset for machine learning based on the preprocessed data and sentiment data. Generative AI is used to generate a delivery evaluation model. This evaluation model has criteria for evaluating delivery quality and efficiency. For example, evaluation criteria include timely delivery and flexibility in customer service.

[1446] Conducting the evaluation

[1447] The server evaluates new delivery data using the generated evaluation model. The evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative evaluation (e.g., "very good"). These evaluation results are used as the basis for strategic decision-making and improvement measures for delivery companies.

[1448] feedback

[1449] The server then notifies the delivery person or the transportation company of the evaluation results. The notification includes a breakdown of the evaluation and suggestions for improvement, and provides feedback such as "Overall rating: 4.0" and "User satisfaction: High."

[1450] Emotional Data Feedback

[1451] The server also provides the delivery person with feedback on the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[1452] Expanding system usage

[1453] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[1454] Monetization

[1455] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from delivery companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[1456] Specific examples

[1457] Processing user reviews

[1458] A user submits a review on a mobile app stating, "Delivery was delayed this time, but the delivery person's service was excellent."

[1459] Determining shipping fees

[1460] The delivery person checks the evaluation results on the terminal and attempts to set a new delivery fee.

[1461] This specific embodiment of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. By incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, delivery personnel and transportation companies can maintain higher levels of customer satisfaction.

[1462] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1463] Step 1: Collect reviews

[1464] After using the delivery service, a user posts a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user may post a review stating, "This delivery was delayed, but the delivery person's response was very good."

[1465] Input: User review after delivery service completed

[1466] Output: Reviews stored in a database

[1467] Specific behavior:

[1468] The user logs into the mobile app and opens an interface to enter a review for the specified delivery.

[1469] Write your review and hit submit.

[1470] The server receives the submitted reviews in real time and stores them in a database.

[1471] Step 2: Preprocessing the data

[1472] The server preprocesses the collected reviews. Specifically, it removes inaccurate data, fills in missing values, and analyzes the review text data. Natural language processing (NLP) technology is used for this analysis to extract keywords such as "delay" and "good service."

[1473] Input: Reviews stored in the database

[1474] Output: Preprocessed review text data

[1475] Specific behavior:

[1476] The server retrieves the collected reviews from the database.

[1477] Applying filtering algorithms to remove fraudulent data and spam.

[1478] Apply methods to impute missing data (e.g., mean imputation).

[1479] The review text is analyzed using natural language processing algorithms to extract important keywords and phrases.

[1480] Step 3: Emotion Recognition

[1481] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[1482] Input: Preprocessed text data of reviews

[1483] Output: Sentiment data categorized and scored

[1484] Specific behavior:

[1485] The server runs the sentiment engine (e.g., sentiment analysis algorithm).

[1486] Using text data as input, the system recognizes the user's emotions (e.g., satisfaction, sense of victimization).

[1487] Converting sentiment data into a numerical score, making this data in a format that can be used in rating models.

[1488] Step 4: Generate the evaluation model

[1489] The server creates a machine learning dataset based on the preprocessed data and sentiment data. Generative AI is then used to generate a delivery evaluation model. This evaluation model contains criteria for evaluating delivery quality and efficiency.

[1490] Input: Preprocessed review text and sentiment data

[1491] Output: Shipping evaluation model

[1492] Specific behavior:

[1493] The server builds a dataset to run machine learning algorithms.

[1494] The dataset includes pre-processed data and scored sentiment data.

[1495] Generative AI is used to generate models from this data to evaluate delivery quality and efficiency.

[1496] Train and validate the models and save the best evaluation model.

[1497] Step 5: Conduct the assessment

[1498] The server evaluates new delivery data using the generated evaluation model, and the evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative rating (e.g., "Very Good").

[1499] Input: New delivery data

[1500] Output: Evaluation results (numerical and qualitative evaluation)

[1501] Specific behavior:

[1502] The server inputs the new delivery data into the rating model.

[1503] Run the model and calculate a rating for each element of the shipping data.

[1504] The evaluation results are output in numerical and qualitative report formats.

[1505] Step 6: Feedback of the evaluation results

[1506] The server notifies the delivery person or the transportation company of the evaluation results, including a breakdown of the evaluation and suggestions for improvement.

[1507] Input: Evaluation result

[1508] Output: Feedback (evaluation results, areas for improvement)

[1509] Specific behavior:

[1510] The server creates a feedback message based on the evaluation results.

[1511] A notification is sent to the delivery person or transportation company's device.

[1512] The terminal receives the notification and performs the function of displaying it to the delivery person.

[1513] Step 7: Feedback of emotional data

[1514] The server also feeds back to the delivery person the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[1515] Input: Emotion data

[1516] Output: Feedback content (emotion data)

[1517] Specific behavior:

[1518] The server retrieves the recognized emotion data and formats it for feedback.

[1519] Feedback including emotional data is sent to the delivery person's device.

[1520] The terminal receives the notification and performs the function of displaying it to the delivery person.

[1521] Step 8: Expanding system usage

[1522] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[1523] Input: Evaluation result

[1524] Output: Shipping fee setting options, registration management results

[1525] Specific behavior:

[1526] The terminal displays the evaluation results and presents delivery fee setting options to the delivery person.

[1527] The delivery person will set the shipping fee based on the options provided.

[1528] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1529] Step 9: Monetization

[1530] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[1531] Input: Shipping fee data

[1532] Output: System usage fee calculation result

[1533] Specific behavior:

[1534] The server takes delivery data as input and executes an algorithm to calculate system usage fees.

[1535] We operate a system that automatically collects calculated fees from transportation companies and individual business owners.

[1536] Store the results of fee collection and perform the necessary processes to maintain and operate the system.

[1537] (Application example 2)

[1538] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1539] Conventional delivery evaluation systems are unable to take user emotions into account, making it difficult to provide fast and accurate evaluations and feedback. Furthermore, they lack a mechanism for comprehensively evaluating delivery personnel's performance and customer satisfaction and providing feedback, preventing improvements in delivery quality. Therefore, there is a need for an evaluation system that includes emotion recognition to improve the overall quality of delivery services.

[1540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1541] In this invention, the server includes means for collecting user reviews, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for feeding back the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from user reviews and using them as part of the rating, and means for providing feedback based on the rating results and emotion data. This enables comprehensive delivery ratings that take user emotions into account, and enables delivery quality to be improved through fast and accurate feedback.

[1542] "User" refers to an individual who uses the Delivery Service and submits a review.

[1543] "Reviews" refer to opinions and impressions posted by users about delivery services.

[1544] "Preprocessing" refers to the process of converting collected reviews into a format suitable for the rating model.

[1545] "Evaluation model" refers to an algorithm generated using generative artificial intelligence to calculate delivery quality and efficiency.

[1546] "Feedback" refers to notifying delivery personnel and transportation companies of the evaluation results and informing them of areas for improvement and the evaluation results.

[1547] "Delivery person" refers to a person who provides delivery services to users.

[1548] "Transportation Company" refers to a company that provides delivery services.

[1549] "Emotion recognition" refers to the technology of analyzing emotions (satisfaction, dissatisfaction, surprise, etc.) from user reviews.

[1550] "Emotion data" refers to data that indicates the user's emotions extracted by emotion recognition.

[1551] "Delivery fee" refers to the fee paid by a user when using a delivery service.

[1552] "Shipping Fee Limit" refers to the maximum shipping fee set based on the shipping terms.

[1553] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[1554] "Independent Business Owner" refers to an individual who registers with the system to provide delivery services.

[1555] "Evaluation criteria" refers to the standards used to evaluate the performance of delivery personnel and transportation companies.

[1556] "Evaluation results" refer to the results of delivery quality and efficiency calculated by the evaluation model.

[1557] The present invention is a system for improving evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. MODES FOR CARRYING OUT THE INVENTION The following describes embodiments of the present invention.

[1558] System configuration

[1559] The system consists of a user, a terminal, and a server. Users use smartphones or PCs to input reviews after delivery services. The terminals are used by delivery personnel and transportation companies and receive evaluation results and areas for improvement as feedback. The server is the core component that collects data, preprocesses it, generates evaluation models, performs evaluations, and provides feedback.

[1560] Hardware and software used

[1561] Hardware:

[1562] Server: An EC2 instance on Amazon Web Services (AWS)

[1563] User device: Smartphone (Android / iOS)

[1564] software:

[1565] Sentiment Analysis: TextBlob (Python library)

[1566] Data preprocessing: scikit-learn (Python library)

[1567] Machine learning model: Support Vector Machine (SVM)

[1568] Data processing and calculation

[1569] The server collects user-entered reviews in real time and stores them in a database. The collected reviews are then preprocessed to convert the text data into an analyzable format. The preprocessed data is then analyzed by an emotion engine to recognize the user's emotional state. This emotion data and the preprocessed reviews are then used to generate a rating model.

[1570] The evaluation model is generated using generative artificial intelligence (generative AI model), and evaluations are performed based on new delivery data. The evaluation results are output as numerical and qualitative evaluations and notified to the terminal. The evaluation results also include areas for improvement, which delivery personnel and transportation companies can use as a reference to improve performance. In addition, user emotional data is notified as feedback, allowing delivery personnel to take specific measures to improve.

[1571] Specific examples

[1572] User review processing example

[1573] A user uses a delivery service and posts a review from a mobile app saying, "This delivery was delayed, but the delivery person's service was excellent." The server stores this review in a database, analyzes the text, and assigns a score of "delay." The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text. The server then inputs the evaluation data and emotion data into an evaluation model to evaluate the delivery person's performance. For example, an evaluation score of "4.2 / 5" is generated. The server notifies the delivery person's device of the evaluation result and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on their next delivery.

[1574] Example prompt statement

[1575] After delivery, a review is posted saying, "This delivery was a little late, but the delivery person's service was excellent." Recognize the user's sentiment from this review and generate a score to evaluate the delivery person's performance.

[1576] This enables comprehensive delivery evaluation that takes into account the user's emotions, and enables delivery quality to be improved through fast and accurate feedback.

[1577] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1578] Step 1:

[1579] After using the delivery service, the user enters a review using a mobile app on their smartphone. The review is sent to the server in real time. Once the review is completed, it is saved in the server's database and becomes available for subsequent processing.

[1580] Step 2:

[1581] The server preprocesses the collected reviews. Specifically, it removes unnecessary characters and symbols from the review text data and converts it into an analyzable format. At this stage, it also removes invalid data and fills in missing values. The input is the reviews entered by the user, and the output is the preprocessed, clean text data.

[1582] Step 3:

[1583] The server inputs the preprocessed reviews into an emotion engine for sentiment analysis. The emotion engine (e.g., TextBlob) recognizes the user's emotion (e.g., satisfied, dissatisfied, etc.) from the review text data. The emotion analysis results are output as a quantitative emotion score.

[1584] Step 4:

[1585] The server generates a rating model using a generative AI model (e.g., support vector machine, SVM) based on the preprocessed reviews and sentiment data. The input is the preprocessed data and sentiment data, and the output is a rating model for the delivery.

[1586] Step 5:

[1587] The server inputs new delivery data into the evaluation model and performs the evaluation, which evaluates the performance of the new delivery. The input is the new delivery data, and the output is an evaluation score and qualitative feedback.

[1588] Step 6:

[1589] The server notifies the terminal of the delivery person or transportation company of the evaluation results. The evaluation results also include areas for improvement, and the delivery person or transportation company uses this feedback to improve their performance. Specifically, the evaluation results and comments are displayed in the application on the terminal.

[1590] Step 7:

[1591] The server also provides emotional feedback, allowing delivery personnel to understand the user's specific emotional state and take measures to improve their next delivery. The emotional data is displayed on the terminal as comments that can help delivery personnel improve their work.

[1592] Step 8:

[1593] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. Delivery people can set delivery fees through a smartphone app based on their own evaluation. The server also sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and the customer's emotional state, and calculates the optimal delivery fee. This is displayed on the terminal, allowing the delivery person to check and adjust it.

[1594] Step 9:

[1595] The server calculates a system usage fee based on the evaluation results and emotion data, and automatically collects it as a percentage of the delivery fee. Fees from delivery companies and individual business owners are automatically calculated and collected via the server's system. Specifically, invoices are generated and sent on the server.

[1596] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1597] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1598] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1599] [Fourth embodiment]

[1600] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1601] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1602] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1603] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1604] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1606] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1607] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1608] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1609] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1610] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1611] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1612] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1613] The present invention aims to improve delivery quality and efficiency by introducing a delivery evaluation system. Specific embodiments for carrying out the present invention will be described below.

[1614] This system consists of a user, a terminal, and a server. The user uses the delivery service and then rates it. Delivery personnel and transportation companies use the terminal to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates rating models, performs the rating, and provides feedback on the results.

[1615] Program processing overview

[1616] 1. Data Collection

[1617] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[1618] The server collects reviews in real time and stores them in a database.

[1619] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1620] 2. Data Preprocessing

[1621] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[1622] 3. Generating the evaluation model

[1623] The server creates a machine learning dataset based on the preprocessed data.

[1624] The server uses artificial intelligence to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[1625] 4. Conducting the evaluation

[1626] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[1627] 5. Feedback of evaluation results

[1628] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[1629] 6. Expanding system usage

[1630] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[1631] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1632] 7. Monetization

[1633] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[1634] Specific examples

[1635] Processing user reviews

[1636] 1. A user actually uses the delivery service and, after the delivery is completed, posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[1637] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[1638] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[1639] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[1640] Determining shipping fees

[1641] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[1642] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions.

[1643] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and calculates the optimal shipping fee.

[1644] This specific form of the present invention enables systematic evaluation and improvement of delivery quality and efficiency, and also realizes flexible pricing and preferential handling fees based on the evaluation, which is expected to optimize the entire delivery service.

[1645] The processing flow will be explained below.

[1646] Step 1:

[1647] A user uses a delivery service and writes a review after delivery is completed.

[1648] Users post reviews and ratings about deliveries using a mobile app or web interface.

[1649] Step 2:

[1650] The server collects user reviews.

[1651] The server stores the reviews in a database in real time.

[1652] Step 3:

[1653] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1654] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[1655] Step 4:

[1656] The server preprocesses the collected data.

[1657] The server normalizes the review text data, removes invalid data, and imputes missing values.

[1658] Step 5:

[1659] The server generates a dataset for machine learning.

[1660] The server extracts features and labels them based on the preprocessed data.

[1661] Step 6:

[1662] The server trains the evaluation model using generative AI.

[1663] The server uses machine learning algorithms to train a rating model for deliveries.

[1664] Step 7:

[1665] The server inputs the new delivery data into the rating model.

[1666] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[1667] Step 8:

[1668] The server generates the evaluation results.

[1669] The server generates evaluation results and improvement suggestions based on the evaluation scores output from the model.

[1670] Step 9:

[1671] The server notifies the delivery person and the transportation company of the evaluation results.

[1672] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[1673] Step 10:

[1674] The terminal displays the evaluation results.

[1675] The terminal (delivery person's smartphone or tablet) displays the evaluation results and improvement suggestions.

[1676] Step 11:

[1677] The terminal provides the delivery person with the option to set delivery fees.

[1678] The terminal displays a delivery fee setting screen to the delivery person, allowing the delivery person to set the delivery fee.

[1679] Step 12:

[1680] The server automatically determines the shipping limit.

[1681] The server automatically sets an appropriate upper limit on delivery charges, taking into account delivery conditions and the delivery person's past evaluations.

[1682] Step 13:

[1683] The delivery person sets the shipping fee.

[1684] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[1685] Step 14:

[1686] The server manages the registration of sole proprietors.

[1687] The server accepts the registration information of the sole proprietor and checks the qualifications based on the evaluation criteria.

[1688] Step 15:

[1689] The server calculates and collects fees for using the system.

[1690] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[1691] Example 1

[1692] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1693] Current delivery services lack a system that accurately evaluates delivery quality and efficiency and provides appropriate feedback to delivery personnel and transportation companies. As a result, it is difficult to improve service quality and efficiency, and user satisfaction declines. In addition, there is a lack of a system that effectively sets appropriate fees and provides preferential treatment for commissions based on evaluations, making it difficult to improve delivery personnel motivation or ensure consistency in service quality.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1695] In this invention, the server includes a means for collecting reviews from users, a means for preprocessing the collected reviews and converting them into a format suitable for evaluation using a natural language processing tool, and a means for creating a machine learning dataset based on the preprocessed data, which enables accurate evaluation of delivery quality and efficiency and provides appropriate feedback.

[1696] "User" refers to a general consumer who uses goods or services.

[1697] "Review" refers to the evaluation or feedback provided by a user after using a product or service.

[1698] "Preprocessing" refers to the process of preparing collected data in a form that can be analyzed and evaluated.

[1699] "Natural language processing tools" refer to software technology for analyzing text data and extracting necessary information.

[1700] A "machine learning dataset" refers to a collection of data used to train a machine learning model.

[1701] "Generative artificial intelligence" refers to programs that have the ability to perform specific tasks using machine learning and deep learning.

[1702] An "evaluation model" refers to an algorithm or system that analyzes data and produces results based on specific evaluation criteria.

[1703] "Delivery Data" means data containing information regarding the progress and results of a delivery.

[1704] "Evaluation Results" refers to the numerical or qualitative evaluations obtained using an evaluation model.

[1705] "Feedback" refers to the process of informing stakeholders of evaluation results and areas for improvement.

[1706] "Fees" refers to the fees paid for use of the Services.

[1707] "Delivery Charge" means the charge payable for delivery of Goods.

[1708] "Maximum shipping fee" refers to the maximum shipping fee set based on the shipping conditions.

[1709] "Sole proprietor" refers to an individual who operates a business as a self-employed person.

[1710] "Evaluation criteria" refers to the standards and indicators used for evaluation.

[1711] "Qualifications" refers to the abilities and conditions necessary to perform a particular role or task.

[1712] The purpose of this invention is to improve delivery quality and efficiency by introducing a delivery evaluation system. This system consists of three entities: a user, a terminal, and a server.

[1713] 1. Data Collection

[1714] After the delivery is complete, the user can leave a review via a mobile app or web interface. For example, the user can write a comment such as "The delivery was 30 minutes later than expected."

[1715] The server collects these reviews in real time and stores them in a database (e.g., MySQL or PostgreSQL). The server is configured to receive and store data using a REST API. The server also collects information about damage and delays from the terminals of delivery companies and delivery personnel. This process is carried out via API, and the latest information is maintained by periodic data transmission from the terminals.

[1716] 2. Data Preprocessing

[1717] The server preprocesses the collected data. Specifically, it analyzes the collected text data using natural language processing tools (e.g., NLTK or SpaCy). This analysis generates the keywords and scoring necessary for evaluation. The server also removes invalid data and completes missing values. For example, invalid data would be data that has clearly been entered incorrectly or is in a non-standard format, and this is eliminated through automatic screening.

[1718] 3. Generating the evaluation model

[1719] The server creates a machine learning dataset based on the preprocessed data. The dataset includes features such as delivery time, rating points, and text reviews. The server then generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch). This model has the ability to train the dataset and evaluate delivery quality and efficiency. The generated model is stored internally on the server for use during evaluation.

[1720] 4. Conducting the evaluation

[1721] The server inputs new delivery data into the evaluation model and calculates the evaluation results. For example, when new delivery data is input into the model, an evaluation score and qualitative comments are automatically generated. The server saves the evaluation results in a database and simultaneously adds them to a notification queue. This process makes the evaluation results available immediately.

[1722] 5. Feedback of evaluation results

[1723] The server notifies the delivery person or transportation company of the evaluation results. This notification is sent instantly using an API or WebSocket. The device receives the notification and displays the evaluation results and areas for improvement to the delivery person. For example, specific feedback such as "Please be punctual" may be included.

[1724] 6. Expanding system usage

[1725] The terminal displays the evaluation results to the delivery person and provides delivery fee setting options. The delivery person adjusts the delivery fee based on the evaluation using the terminal UI. The server manages the registration of individual business owners and checks their eligibility based on the evaluation criteria. For example, a warning message is displayed if the evaluation criteria are not met during new registration.

[1726] 7. Monetization

[1727] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This is done using payment gateways such as credit cards and bank transfers. The collected fee is transferred to the operator's account and used to cover system maintenance costs.

[1728] Specific examples

[1729] Processing user reviews

[1730] 1. After completing delivery, the user posts a review from the mobile app stating that "the delivery was 30 minutes later than expected."

[1731] 2. The server stores this review in a database, analyzes the text (e.g., SpaCy), and scores it as "delayed."

[1732] 3. The server inputs the analysis results into the evaluation model and evaluates the delivery person's performance. For example, an evaluation score of "3.5 / 5" is generated.

[1733] 4. The server notifies the delivery person's device of the evaluation results and suggestions for improvement, and the delivery person uses this information to improve their performance on the next delivery.

[1734] Determining shipping fees

[1735] 1. The terminal displays the evaluation results to the delivery person, giving them an "Overall rating: 4.0" and suggestions for improvement.

[1736] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on the AI's suggestions. For example, the system may display a message saying, "We recommend setting the fee between 1,000 and 1,200 yen."

[1737] 3. The server sets a maximum delivery fee based on delivery conditions (distance, time of day, etc.) and calculates the optimal delivery fee. For example, dynamic pricing is possible, such as setting an additional fee for late-night deliveries.

[1738] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1739] Step 1:

[1740] The user writes a review after the delivery is completed.

[1741] Specifically, users use a mobile app or web interface to post comments such as "Delivery was 30 minutes later than expected."

[1742] Input: User-entered text review

[1743] Output: Text review data sent to the server

[1744] Step 2:

[1745] The server receives reviews from users and stores them in a database in real time.

[1746] Specifically, the server receives data via a REST API and stores it in a MySQL or PostgreSQL database, validating the data format during this process.

[1747] Input: User-submitted text review

[1748] Output: Review information stored in a database

[1749] Step 3:

[1750] The server preprocesses the collected data.

[1751] Specifically, we use natural language processing tools (e.g., NLTK and SpaCy) to analyze text data and convert it into a format suitable for evaluation. We also remove invalid data and impute missing values.

[1752] Input: Review information stored in the database

[1753] Output: Preprocessed text data

[1754] Step 4:

[1755] The server creates a machine learning dataset based on the preprocessed data.

[1756] Specifically, collected features such as review information, delivery time, and rating points are incorporated into a dataset, which generates a dataset in a format that can be used for rating model training.

[1757] Input: Preprocessed text data

[1758] Output: Dataset for machine learning

[1759] Step 5:

[1760] The server generates an evaluation model using generative artificial intelligence (e.g., TensorFlow or PyTorch).

[1761] Specifically, the system uses preprocessed data to train a model and generate an evaluation model capable of evaluating delivery quality and efficiency. The generated model is stored on the server.

[1762] Input: Machine learning dataset

[1763] Output: Evaluation model

[1764] Step 6:

[1765] The server inputs new delivery data into the rating model, calculates and outputs the rating results.

[1766] Specifically, new delivery data (e.g., delivery time, user ratings) is input into the evaluation model, and evaluation scores and improvement suggestions are automatically generated.

[1767] Input: New shipping data

[1768] Output: Evaluation results (score, improvement suggestions)

[1769] Step 7:

[1770] The server notifies the delivery person or transportation company of the evaluation results.

[1771] Specifically, the evaluation results are instantly sent via API and WebSocket and displayed on the device, including not only the evaluation score but also points for improvement.

[1772] Input: Evaluation result

[1773] Output: Evaluation results displayed on the terminal of the delivery person or transportation company

[1774] Step 8:

[1775] The terminal provides a shipping fee setting option, and the delivery person sets the shipping fee.

[1776] Specifically, a pricing screen based on AI suggestions is displayed on the device's UI, and the delivery person inputs the optimal delivery fee. Based on this, the server calculates the appropriate fee and sets an upper limit.

[1777] Input: Shipping fee information set on the device

[1778] Output: Shipping fee settings reflected in the system

[1779] Step 9:

[1780] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1781] Specifically, if the evaluation criteria are not met during new registration, a warning message will be displayed. The server will check the qualifications of the sole proprietor based on the evaluation score and decide whether to approve the registration.

[1782] Input: New registration information

[1783] Output: Credential check results and warning messages (if necessary)

[1784] Step 10:

[1785] The server calculates and collects the system usage fee as a percentage of the shipping fee.

[1786] Specifically, fees are automatically collected from transportation companies and individual business owners through a credit card payment gateway and transferred to the operator's account.

[1787] Input: Shipping fee information, handling fee percentage

[1788] Output: Collected fees and transfer details

[1789] (Application example 1)

[1790] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1791] In conventional food delivery services, evaluation of delivery quality and efficiency is subjective, and areas for improvement are not clearly communicated, making it difficult to improve overall service quality. Furthermore, delivery fees are set based on the delivery person's experience and intuition, which can lead to inappropriate pricing, resulting in reduced revenue and customer satisfaction. Furthermore, evaluation criteria are vague, making it difficult for delivery people and transportation companies to obtain useful feedback to optimize their performance. These issues need to be resolved.

[1792] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1793] In this invention, the server includes a means for collecting user reviews, a means for preprocessing the collected reviews and inputting them into a rating model, and a means for generating a delivery rating model using generative artificial intelligence. This enables objective evaluation of delivery quality and efficiency. The server further includes a means for inputting new delivery data into the rating model and outputting the rating results, a means for notifying the user of the rating results and providing delivery fee setting options, and a means for providing feedback on the rating results and suggesting areas for performance improvement. This allows delivery personnel and transportation companies to objectively evaluate their performance and identify specific areas for improvement based on the feedback. The server also includes a means for automatically determining a maximum delivery fee based on delivery conditions, a means for offering preferential commissions to delivery personnel who receive a certain rating or higher based on the rating results, a means for evaluating delivery personnel based on delivery data from the food delivery service, and a means for providing feedback on specific areas for improvement to delivery personnel who do not meet certain rating standards. This enables appropriate delivery fee settings and fair preferential commissions, which is expected to improve the overall quality of the service.

[1794] "User" refers to a person who uses the food delivery service to order and receive meals.

[1795] "Reviews" refer to opinions and impressions provided by users after delivery is completed.

[1796] "Preprocessing" refers to the process of analyzing collected reviews and converting them into a format suitable for the rating model.

[1797] An "evaluation model" refers to a machine learning model that has criteria for evaluating delivery quality and efficiency.

[1798] "Delivery Data" refers to all data related to delivery (delivery time, delay information, user reviews, etc.).

[1799] "Evaluation results" refer to the numerical values ​​and qualitative indicators output by the evaluation model that indicate the quality and efficiency of delivery.

[1800] "Feedback" refers to the points for improvement and evaluation provided based on the evaluation results.

[1801] "Performance" refers to the efficiency and quality of delivery work performed by delivery personnel and transportation companies.

[1802] "Delivery fee" refers to the fee paid by the user when using the delivery service.

[1803] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[1804] "Food delivery service" refers to a service that allows users to order food and drinks online and have them delivered to a specified location.

[1805] "Generative AI" refers to AI that uses machine learning and data analysis to perform specific tasks.

[1806] The present invention aims to evaluate delivery quality and efficiency in food delivery services and improve the overall quality of the service. Specific embodiments for carrying out the present invention will be described below.

[1807] System Configuration

[1808] The system of the present invention is mainly composed of a server, a terminal (such as a smartphone), and a user.

[1809] server:

[1810] The server is the core component that collects data, preprocesses it, generates evaluation models, performs the evaluation, and provides feedback on the results. Specific software used includes Python, Pandas, and Scikit-learn.

[1811] Device:

[1812] The terminal is a device used by delivery personnel and transportation companies to check evaluation results, set delivery fees, and receive feedback. Specifically, this is realized as a smartphone app.

[1813] User:

[1814] Users are people who use food delivery services and post reviews after delivery is complete.

[1815] System Features

[1816] 1. Data Collection:

[1817] When a user uses a food delivery service and the delivery is completed, the user enters a review through the mobile app. The server collects these reviews in real time and stores them in a database.

[1818] 2. Data Preprocessing:

[1819] The server preprocesses the collected data. It analyzes the review text data and converts it into a format suitable for the rating model. For example, if a review says "30 minutes late," it will be scored as "late."

[1820] 3. Generate the evaluation model:

[1821] The server creates a machine learning dataset based on the preprocessed data, generates a random forest model using Scikit-learn, and builds a delivery evaluation model.

[1822] 4. Conducting the evaluation:

[1823] New delivery data is input into the evaluation model, and the evaluation results are calculated. The evaluation results are output as numerical or qualitative evaluations, and the server stores these results.

[1824] 5. Feedback of evaluation results:

[1825] The server then notifies the delivery person or the transportation company of the evaluation results. For example, the delivery person's device may receive feedback such as, "Next time, try to be more punctual."

[1826] 6. Set up shipping rates:

[1827] Based on the evaluation results, the delivery person sets the delivery fee on their smartphone. The server calculates the upper limit of the delivery fee based on delivery conditions (distance, time zone, etc.) and proposes the optimal delivery fee.

[1828] Specific examples

[1829] For example, if a user posts a review on a mobile app stating that "the delivery was 30 minutes later than scheduled," the server stores this review in a database and uses text analysis to score it as "delayed." This data is then input into a rating model, which rates the delivery person's performance as "3.5 / 5." This rating result is sent to the delivery person's smartphone, and feedback such as "Please be punctual next time" is displayed. The delivery person can also adjust the delivery fee based on this result.

[1830] Example prompt sentence:

[1831] "User review: 30 minutes late. Please analyze the cause of the delivery person's delay and provide a rating."

[1832] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1833] Step 1: User submits review after delivery is complete.

[1834] Input: A review a user leaves on a food delivery app (e.g., "Delivery was 30 minutes later than expected").

[1835] Output: User submitted reviews are sent to the server through the app and stored in a database.

[1836] Specific operation: The user uses the smartphone app to write down their thoughts and opinions after the delivery is completed and presses the submit button. The app then sends the review to the server, which records the review in a database.

[1837] Step 2: The server preprocesses the collected reviews.

[1838] Input: User reviews collected in step 1.

[1839] Output: The text data is converted into a format suitable for the evaluation model (e.g., scored as "delay").

[1840] How it works: The server uses Python and Pandas to parse the review text data, checking for the presence of keywords (e.g., "delay"), and then converts the results into scores in a format suitable for the rating model.

[1841] Step 3: The server generates the evaluation model.

[1842] Input: A preprocessed dataset (e.g., a dataset of "delay" scores).

[1843] Output: A delivery evaluation model (e.g., a random forest model).

[1844] How it works: The server uses Scikit-learn to generate an evaluation model, such as a random forest model, from the preprocessed dataset. The model has criteria for evaluating delivery quality and efficiency.

[1845] Step 4: The server inputs the new delivery data into the rating model and performs the rating.

[1846] Input: New delivery data (e.g., new user reviews and their preprocessing results).

[1847] Output: Evaluation result (e.g., delivery driver performance score "3.5 / 5").

[1848] Specific operation: The server inputs new delivery data into the evaluation model to evaluate the delivery person's performance. The evaluation results are generated as numerical values ​​and stored in the database.

[1849] Step 5: The server notifies the delivery person's terminal of the evaluation result.

[1850] Input: Evaluation result (e.g., performance score "3.5 / 5" with specific feedback).

[1851] Output: A notification is sent to the delivery person's smartphone (e.g., "Please be punctual next time").

[1852] Specific operation: The server compiles the evaluation results and generates a notification message. It then sends the message to the delivery person through a smartphone app. The delivery person receives the notification and checks the feedback on the app.

[1853] Step 6: The delivery person sets the shipping fee based on the evaluation results.

[1854] Input: Evaluation results and AI-generated shipping fee suggestions.

[1855] Output: New shipping price configuration (e.g. suggested optimal shipping price).

[1856] Specific operation: The delivery person opens the delivery fee setting screen on their smartphone app and determines the fee based on the AI's suggestions. The server calculates the maximum delivery fee based on delivery conditions (distance, time zone, etc.) and displays the optimal delivery fee. The delivery person sets the delivery fee according to the suggestions and sends it to the server.

[1857] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1858] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. Below, we will explain in detail how each component works together to realize the present invention.

[1859] This system consists of a user, a terminal, and a server. Users use delivery services and then rate them. Delivery personnel and shipping companies use their terminals to receive feedback from the system, set delivery fees, and check the rating results. The server is the core component that collects data, preprocesses it, generates a rating model, performs the rating, and provides feedback on the results. Furthermore, an emotion engine is used to recognize emotions from user reviews and use them as part of the rating.

[1860] Program processing overview

[1861] 1. Data Collection

[1862] Once delivery is complete, the user can leave a review via the mobile app or web interface.

[1863] The server collects reviews in real time and stores them in a database.

[1864] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1865] 2. Data Preprocessing

[1866] The server preprocesses the collected data, analyzes the review text data, and converts it into a format suitable for evaluation. It also removes invalid data and completes missing values.

[1867] 3. Emotion recognition

[1868] The server uses an emotion engine to recognize the user's emotion (e.g., satisfaction, dissatisfaction, surprise, etc.) from the review text data.

[1869] The emotion engine categorizes and scores the emotional data it recognizes.

[1870] 4. Generating the evaluation model

[1871] The server creates a machine learning dataset based on the preprocessed data and emotion data.

[1872] The server uses generative AI to generate a delivery evaluation model, which has criteria for evaluating delivery quality and efficiency.

[1873] 5. Conducting the evaluation

[1874] The server inputs new delivery data into the rating model and calculates the rating result, which is output as a numerical or qualitative rating.

[1875] 6. Feedback of evaluation results

[1876] The server notifies the delivery person or transportation company of the evaluation results, which include points for improvement and are provided as feedback.

[1877] 7. Emotional Data Feedback

[1878] The server provides the delivery person with the user's emotional data recognized by the emotion engine as feedback, allowing the delivery person to understand the user's specific emotional state and take measures to improve the situation.

[1879] 8. Expanding System Usage

[1880] The terminal displays the evaluation results to the delivery person and provides the option to set the delivery fee, which the delivery person can then set based on their own evaluation.

[1881] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[1882] 9. Monetization

[1883] The server calculates the system usage fee as a few percent of the shipping fee and automatically collects it from shipping companies and individual business owners.

[1884] Specific examples

[1885] Processing user reviews

[1886] 1. A user uses a delivery service and, after completing the delivery, posts a review on the mobile app stating, "This delivery was delayed, but the delivery person's service was excellent."

[1887] 2. The server stores this review in a database, analyzes the text, and scores it as "delayed."

[1888] 3. The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text.

[1889] 4. The server inputs this evaluation data and emotion data into the evaluation model to evaluate the delivery driver's performance. For example, an evaluation score of "4.2 / 5" is generated.

[1890] 5. The server notifies the delivery person's device of the evaluation results and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on the next delivery.

[1891] Determining shipping fees

[1892] 1. The terminal displays the evaluation results to the delivery person and provides suggestions for improvement, with an overall rating of 4.0 and high user satisfaction.

[1893] 2. The delivery person opens the delivery fee setting screen on their device and determines the fee based on AI suggestions and emotional data.

[1894] 3. The server sets a maximum shipping fee based on delivery conditions (distance, time zone, etc.) and the customer's emotional state, and calculates the optimal shipping fee.

[1895] This specific implementation of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. Furthermore, by incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, the present invention enables flexible pricing and preferential commission rates based on reviews, allowing delivery personnel and transportation companies to maintain higher levels of customer satisfaction. This is expected to lead to the optimization of the entire delivery service.

[1896] The processing flow will be explained below.

[1897] Step 1:

[1898] A user uses a delivery service and writes a review after delivery is completed.

[1899] Users use a mobile app or web interface to post reviews and ratings about deliveries.

[1900] Step 2:

[1901] The server collects user reviews.

[1902] The server stores the reviews in a database in real time.

[1903] Step 3:

[1904] The server collects information on property damage and delays from the terminals of transportation companies and delivery personnel.

[1905] The server obtains report data on damage and delays that occur during delivery from the terminals of transportation companies and delivery personnel.

[1906] Step 4:

[1907] The server preprocesses the collected data.

[1908] The server normalizes the review text data, removes invalid data, and imputes missing values.

[1909] Step 5:

[1910] The server uses an emotion engine to recognize emotions from user reviews.

[1911] The server inputs the review text data into an emotion engine, which recognizes and classifies emotions such as "satisfied," "dissatisfied," and "surprise."

[1912] Step 6:

[1913] The server scores the output of the emotion engine.

[1914] The server converts the output of the emotion engine into numerical data and formats it in a way that can be used in the evaluation model.

[1915] Step 7:

[1916] The server combines the emotion data and preprocessed data to generate a dataset for machine learning.

[1917] The server combines the pre-processed data with the emotion data from the emotion engine to create a dataset for machine learning.

[1918] Step 8:

[1919] The server trains the evaluation model using generative AI.

[1920] The server uses machine learning algorithms to generate a delivery evaluation model, with the training dataset including sentiment data.

[1921] Step 9:

[1922] The server inputs the new delivery data into the rating model.

[1923] The server inputs the data for each delivery job into the evaluation model and calculates the evaluation results.

[1924] Step 10:

[1925] The server generates the evaluation results.

[1926] The server generates evaluation results regarding delivery quality and efficiency based on the evaluation scores output from the model, which also include feedback based on the user's emotions.

[1927] Step 11:

[1928] The server notifies the delivery person and the transportation company of the evaluation results.

[1929] The server sends the evaluation results to the delivery person's terminal and the transportation company's management system.

[1930] Step 12:

[1931] The terminal displays the evaluation results.

[1932] The terminal (delivery worker's smartphone or tablet) displays the user's emotional feedback along with the evaluation score. This screen also includes suggestions for improvement.

[1933] Step 13:

[1934] The terminal provides the delivery person with the option to set delivery fees.

[1935] The terminal displays a delivery fee setting screen to the delivery person, allowing them to set fees based on evaluation results and emotional data.

[1936] Step 14:

[1937] The server automatically determines the shipping limit.

[1938] The server automatically determines an appropriate upper limit for delivery charges, taking into account delivery conditions, past evaluations of delivery personnel, and recognized user emotional data.

[1939] Step 15:

[1940] The delivery person sets the shipping fee.

[1941] The delivery person sets the delivery fee according to the instructions on the terminal and notifies the server.

[1942] Step 16:

[1943] The server manages the registration of sole proprietors.

[1944] The server accepts the registration information of the sole proprietor and verifies the qualifications based on the evaluation criteria.

[1945] Step 17:

[1946] The server calculates and collects fees for using the system.

[1947] The server calculates a few percent of the shipping fee as a commission fee and automatically collects it from shipping companies and individual business owners.

[1948] Example 2

[1949] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1950] Current delivery rating systems only collect user reviews but do not fully utilize their sentiment and specific feedback. Furthermore, there are insufficient means to directly provide feedback on the ratings to delivery personnel and shipping companies, slowing down improvements in delivery quality and efficiency. The current system also has limited functionality for setting delivery fees and preferential handling fees, and the management of setting delivery fee caps and verifying the qualifications of individual business owners is complex. It is necessary to resolve these issues and effectively utilize user reviews and sentiment data to improve overall delivery quality and customer satisfaction.

[1951] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1952] In this invention, the server includes means for collecting reviews from users, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for providing feedback on the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from text data of reviews and using the emotion data as part of the evaluation, and means for evaluating user satisfaction based on the emotion data and providing the result to delivery personnel. This makes it possible to effectively utilize user emotions and specific feedback to improve overall delivery quality and customer satisfaction.

[1953] "Means for collecting reviews from users" refers to functionality for collecting reviews provided by users via a mobile app or web interface.

[1954] The "means for preprocessing collected reviews and inputting them into the evaluation model" is a function for analyzing collected reviews, converting them into a format suitable for evaluation, and inputting them into the evaluation model.

[1955] "Means for generating a delivery evaluation model using generative artificial intelligence" is a function that uses generative AI technology to build a model for evaluating the quality and efficiency of delivery.

[1956] The "means for evaluating new delivery data using an evaluation model and outputting the evaluation results" is a function that inputs newly collected delivery data into the aforementioned evaluation model and outputs the results as numerical or qualitative evaluations.

[1957] "Means of providing feedback on evaluation results and improving the performance of delivery personnel and transportation companies" refers to a function that notifies the evaluation results to delivery personnel and transportation companies and provides suggestions and actions to improve performance based on the results.

[1958] "Means for recognizing emotions from review text data and using the emotional data as part of the evaluation" is a function that analyzes the review text to recognize the user's emotions and uses the emotional data as part of the evaluation criteria.

[1959] The "means for evaluating user satisfaction based on emotional data and providing the results to the delivery person" is a function that quantifies the user's satisfaction based on the emotionally recognized data and provides the results as feedback to the delivery person.

[1960] The present invention aims to further improve the evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. This system is composed of a user, a terminal, and a server. How each element works together to realize the present invention is described in detail below.

[1961] Data collection

[1962] After using a delivery service, users can enter a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user might post a review stating, "This delivery was delayed, but the delivery person's response was excellent."

[1963] The server collects these user reviews in real time and stores them in a database. It also collects information from the terminals of delivery companies and delivery personnel, and integrates data on property damage and delays into the database.

[1964] Data Preprocessing

[1965] The server preprocesses the collected data, specifically removing inaccurate data, filling in missing values, and analyzing the review text data. This analysis uses natural language processing (NLP) technology to extract keywords such as "delay" and "good service."

[1966] emotion recognition

[1967] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[1968] Generating an evaluation model

[1969] The server creates a dataset for machine learning based on the preprocessed data and sentiment data. Generative AI is used to generate a delivery evaluation model. This evaluation model has criteria for evaluating delivery quality and efficiency. For example, evaluation criteria include timely delivery and flexibility in customer service.

[1970] Conducting the evaluation

[1971] The server evaluates new delivery data using the generated evaluation model. The evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative evaluation (e.g., "very good"). These evaluation results are used as the basis for strategic decision-making and improvement measures for delivery companies.

[1972] feedback

[1973] The server then notifies the delivery person or the transportation company of the evaluation results. The notification includes a breakdown of the evaluation and suggestions for improvement, and provides feedback such as "Overall rating: 4.0" and "User satisfaction: High."

[1974] Emotional Data Feedback

[1975] The server also provides the delivery person with feedback on the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[1976] Expanding system usage

[1977] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[1978] Monetization

[1979] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from delivery companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[1980] Specific examples

[1981] Processing user reviews

[1982] A user submits a review on a mobile app stating, "Delivery was delayed this time, but the delivery person's service was excellent."

[1983] Determining shipping fees

[1984] The delivery person checks the evaluation results on the terminal and attempts to set a new delivery fee.

[1985] This specific embodiment of the present invention allows for systematic evaluation and improvement of delivery quality and efficiency. By incorporating user sentiment based on reviews into evaluations and providing feedback based on those sentiments, delivery personnel and transportation companies can maintain higher levels of customer satisfaction.

[1986] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1987] Step 1: Collect reviews

[1988] After using the delivery service, a user posts a review through a mobile app or web interface. This review includes information about the delivery time, contents of the delivery, and the delivery person's response. For example, a user may post a review stating, "This delivery was delayed, but the delivery person's response was very good."

[1989] Input: User review after delivery service completed

[1990] Output: Reviews stored in a database

[1991] Specific behavior:

[1992] The user logs into the mobile app and opens an interface to enter a review for the specified delivery.

[1993] Write your review and hit submit.

[1994] The server receives the submitted reviews in real time and stores them in a database.

[1995] Step 2: Preprocessing the data

[1996] The server preprocesses the collected reviews. Specifically, it removes inaccurate data, fills in missing values, and analyzes the review text data. Natural language processing (NLP) technology is used for this analysis to extract keywords such as "delay" and "good service."

[1997] Input: Reviews stored in the database

[1998] Output: Preprocessed review text data

[1999] Specific behavior:

[2000] The server retrieves the collected reviews from the database.

[2001] Applying filtering algorithms to remove fraudulent data and spam.

[2002] Apply methods to impute missing data (e.g., mean imputation).

[2003] The review text is analyzed using natural language processing algorithms to extract important keywords and phrases.

[2004] Step 3: Emotion Recognition

[2005] The server uses an emotion engine to recognize user emotions from the review text data, such as "satisfied," "dissatisfied," and "surprised." The recognized emotion data is categorized and used in the rating model as a numerical score.

[2006] Input: Preprocessed text data of reviews

[2007] Output: Sentiment data categorized and scored

[2008] Specific behavior:

[2009] The server runs the sentiment engine (e.g., sentiment analysis algorithm).

[2010] Using text data as input, the system recognizes the user's emotions (e.g., satisfaction, sense of victimization).

[2011] Converting sentiment data into a numerical score, making this data in a format that can be used in rating models.

[2012] Step 4: Generate the evaluation model

[2013] The server creates a machine learning dataset based on the preprocessed data and sentiment data. Generative AI is then used to generate a delivery evaluation model. This evaluation model contains criteria for evaluating delivery quality and efficiency.

[2014] Input: Preprocessed review text and sentiment data

[2015] Output: Shipping evaluation model

[2016] Specific behavior:

[2017] The server builds a dataset to run machine learning algorithms.

[2018] The dataset includes pre-processed data and scored sentiment data.

[2019] Generative AI is used to generate models from this data to evaluate delivery quality and efficiency.

[2020] Train and validate the models and save the best evaluation model.

[2021] Step 5: Conduct the assessment

[2022] The server evaluates new delivery data using the generated evaluation model, and the evaluation results are output as a numerical value (e.g., "4.2 / 5") or a qualitative rating (e.g., "Very Good").

[2023] Input: New delivery data

[2024] Output: Evaluation results (numerical and qualitative evaluation)

[2025] Specific behavior:

[2026] The server inputs the new delivery data into the rating model.

[2027] Run the model and calculate a rating for each element of the shipping data.

[2028] The evaluation results are output in numerical and qualitative report formats.

[2029] Step 6: Feedback of the evaluation results

[2030] The server notifies the delivery person or the transportation company of the evaluation results, including a breakdown of the evaluation and suggestions for improvement.

[2031] Input: Evaluation result

[2032] Output: Feedback (evaluation results, areas for improvement)

[2033] Specific behavior:

[2034] The server creates a feedback message based on the evaluation results.

[2035] A notification is sent to the delivery person or transportation company's device.

[2036] The terminal receives the notification and performs the function of displaying it to the delivery person.

[2037] Step 7: Feedback of emotional data

[2038] The server also feeds back to the delivery person the user's emotional data recognized by the emotion engine, allowing the delivery person to understand the user's specific emotions and use this information to improve their performance on the next delivery.

[2039] Input: Emotion data

[2040] Output: Feedback content (emotion data)

[2041] Specific behavior:

[2042] The server retrieves the recognized emotion data and formats it for feedback.

[2043] Feedback including emotional data is sent to the delivery person's device.

[2044] The terminal receives the notification and performs the function of displaying it to the delivery person.

[2045] Step 8: Expanding system usage

[2046] The terminal provides delivery personnel with delivery fee setting options based on the evaluation results. Delivery personnel can set appropriate fees based on their own evaluation. In addition, the server manages the registration of individual business owners and verifies their qualifications based on the evaluation criteria.

[2047] Input: Evaluation result

[2048] Output: Shipping fee setting options, registration management results

[2049] Specific behavior:

[2050] The terminal displays the evaluation results and presents delivery fee setting options to the delivery person.

[2051] The delivery person will set the shipping fee based on the options provided.

[2052] The server manages the registration of sole proprietors and checks their qualifications based on evaluation criteria.

[2053] Step 9: Monetization

[2054] The server calculates the system usage fee as a percentage of the delivery fee and automatically collects it from shipping companies and individual business owners. This fee is used to cover the maintenance and operation costs of the system.

[2055] Input: Shipping fee data

[2056] Output: System usage fee calculation result

[2057] Specific behavior:

[2058] The server takes delivery data as input and executes an algorithm to calculate system usage fees.

[2059] We operate a system that automatically collects calculated fees from transportation companies and individual business owners.

[2060] Store the results of fee collection and perform the necessary processes to maintain and operate the system.

[2061] (Application example 2)

[2062] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2063] Conventional delivery evaluation systems are unable to take user emotions into account, making it difficult to provide fast and accurate evaluations and feedback. Furthermore, they lack a mechanism for comprehensively evaluating delivery personnel's performance and customer satisfaction and providing feedback, preventing improvements in delivery quality. Therefore, there is a need for an evaluation system that includes emotion recognition to improve the overall quality of delivery services.

[2064] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2065] In this invention, the server includes means for collecting user reviews, means for preprocessing the collected reviews and inputting them into a rating model, means for generating a delivery rating model using generative artificial intelligence, means for evaluating new delivery data using the rating model and outputting the rating results, means for feeding back the rating results and improving the performance of delivery personnel and transportation companies, means for recognizing emotions from user reviews and using them as part of the rating, and means for providing feedback based on the rating results and emotion data. This enables comprehensive delivery ratings that take user emotions into account, and enables delivery quality to be improved through fast and accurate feedback.

[2066] "User" refers to an individual who uses the Delivery Service and submits a review.

[2067] "Reviews" refer to opinions and impressions posted by users about delivery services.

[2068] "Preprocessing" refers to the process of converting collected reviews into a format suitable for the rating model.

[2069] "Evaluation model" refers to an algorithm generated using generative artificial intelligence to calculate delivery quality and efficiency.

[2070] "Feedback" refers to notifying delivery personnel and transportation companies of the evaluation results and informing them of areas for improvement and the evaluation results.

[2071] "Delivery person" refers to a person who provides delivery services to users.

[2072] "Transportation Company" refers to a company that provides delivery services.

[2073] "Emotion recognition" refers to the technology of analyzing emotions (satisfaction, dissatisfaction, surprise, etc.) from user reviews.

[2074] "Emotion data" refers to data that indicates the user's emotions extracted by emotion recognition.

[2075] "Delivery fee" refers to the fee paid by a user when using a delivery service.

[2076] "Shipping Fee Limit" refers to the maximum shipping fee set based on the shipping terms.

[2077] "Fees" refer to fees collected from transportation companies and individual business owners for using the system.

[2078] "Independent Business Owner" refers to an individual who registers with the system to provide delivery services.

[2079] "Evaluation criteria" refers to the standards used to evaluate the performance of delivery personnel and transportation companies.

[2080] "Evaluation results" refer to the results of delivery quality and efficiency calculated by the evaluation model.

[2081] The present invention is a system for improving evaluation of delivery quality and efficiency by combining an emotion engine with a delivery evaluation system. MODES FOR CARRYING OUT THE INVENTION The following describes embodiments of the present invention.

[2082] System configuration

[2083] The system consists of a user, a terminal, and a server. Users use smartphones or PCs to input reviews after delivery services. The terminals are used by delivery personnel and transportation companies and receive evaluation results and areas for improvement as feedback. The server is the core component that collects data, preprocesses it, generates evaluation models, performs evaluations, and provides feedback.

[2084] Hardware and software used

[2085] Hardware:

[2086] Server: An EC2 instance on Amazon Web Services (AWS)

[2087] User device: Smartphone (Android / iOS)

[2088] software:

[2089] Sentiment Analysis: TextBlob (Python library)

[2090] Data preprocessing: scikit-learn (Python library)

[2091] Machine learning model: Support Vector Machine (SVM)

[2092] Data processing and calculation

[2093] The server collects user-entered reviews in real time and stores them in a database. The collected reviews are then preprocessed to convert the text data into an analyzable format. The preprocessed data is then analyzed by an emotion engine to recognize the user's emotional state. This emotion data and the preprocessed reviews are then used to generate a rating model.

[2094] The evaluation model is generated using generative artificial intelligence (generative AI model), and evaluations are performed based on new delivery data. The evaluation results are output as numerical and qualitative evaluations and notified to the terminal. The evaluation results also include areas for improvement, which delivery personnel and transportation companies can use as a reference to improve performance. In addition, user emotional data is notified as feedback, allowing delivery personnel to take specific measures to improve.

[2095] Specific examples

[2096] User review processing example

[2097] A user uses a delivery service and posts a review from a mobile app saying, "This delivery was delayed, but the delivery person's service was excellent." The server stores this review in a database, analyzes the text, and assigns a score of "delay." The server uses an emotion engine to recognize "satisfied" and "slightly dissatisfied" from the review text. The server then inputs the evaluation data and emotion data into an evaluation model to evaluate the delivery person's performance. For example, an evaluation score of "4.2 / 5" is generated. The server notifies the delivery person's device of the evaluation result and improvement suggestions that reflect the satisfaction recognized by the emotion engine, and the delivery person uses this information to improve their performance on their next delivery.

[2098] Example prompt statement

[2099] After delivery, a review is posted saying, "This delivery was a little late, but the delivery person's service was excellent." Recognize the user's sentiment from this review and generate a score to evaluate the delivery person's performance.

[2100] This enables comprehensive delivery evaluation that takes into account the user's emotions, and enables delivery quality to be improved through fast and accurate feedback.

[2101] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2102] Step 1:

[2103] After using the delivery service, the user enters a review using a mobile app on their smartphone. The review is sent to the server in real time. Once the review is completed, it is saved in the server's database and becomes available for subsequent processing.

[2104] Step 2:

[2105] The server preprocesses the collected reviews. Specifically, it removes unnecessary characters and symbols from the review text data and converts it into an analyzable format. At this stage, it also removes invalid data and fills in missing values. The input is the reviews entered by the user, and the output is the preprocessed, clean text data.

[2106] Step 3:

[2107] The server inputs the preprocessed reviews into an emotion engine for sentiment analysis. The emotion engine (e.g., TextBlob) recognizes the user's emotion (e.g., satisfied, dissatisfied, etc.) from the review text data. The emotion an...

Claims

1. a means for collecting reviews from users; A means for preprocessing the collected reviews and inputting them into a rating model; A means for generating a delivery evaluation model using generative artificial intelligence; means for evaluating new delivery data using the evaluation model and outputting the evaluation result; A system that includes a means to provide feedback on evaluation results and improve the performance of delivery personnel and transportation companies.

2. a means for allowing a delivery person to set a delivery fee based on the evaluation result; means for automatically determining a shipping fee cap based on shipping terms; The system according to claim 1, further comprising means for giving preferential treatment to delivery personnel with a certain evaluation or higher based on the evaluation results.

3. A means of managing the registration of sole proprietors and verifying their eligibility based on assessment criteria; The system according to claim 1, further comprising means for calculating and collecting a fee for using the system as a percentage of a delivery charge.

Images