System

Abstract

A system is provided.SOLUTION: This system is provided with a means for recognizing a number plate, a means for recording the parking start time and end time of a vehicle, a means for performing charging based on parking time and a system using artificial intelligence for interfacing those means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In apartment complexes, the distance from the parking lot to the living space is often long, creating a need for temporary parking. Due to a lack of suitable parking spaces, residents often park in inappropriate locations, resulting in frequent instances of poor manners. This results in problems such as nuisance to other residents and neighbors, and an increased burden on management companies. Furthermore, the complicated process of measuring temporary parking time and managing charges creates additional challenges. [Means for solving the problem]

[0005] The present invention includes a system that uses artificial intelligence to interface with license plate recognition, a system that records the start and end times of vehicle parking, a system that charges based on parking time, and a system that uses artificial intelligence to interface with these systems. The system also includes a system that compares license plate information with a resident database to determine whether the vehicle belongs to the resident, and a system that provides stored parking records and video data in response to inquiries, thereby improving convenience for residents and reducing the burden on management companies. In particular, by including a system that charges free parking for up to 20 minutes and charges in five-minute increments for parking over 21 minutes, the system can flexibly accommodate temporary parking needs while preventing inappropriate parking.

[0006] A "means for recognizing license plates" is a device, technology, or system that has the function of acquiring a vehicle's license plate as image data, analyzing it, and extracting it as text data.

[0007] A "means for recording vehicle parking start and end times" is a device, technology, or system that automatically detects the time a vehicle enters and leaves a parking space and records these times in a database.

[0008] A "means for charging based on parking time" is a device, technology, or system that has the function of calculating a usage fee in accordance with specified conditions based on the recorded parking time and charging that fee.

[0009] An "artificial intelligence system" is a system that includes a computer program and its execution environment that utilizes machine learning and data analysis techniques to automatically perform tasks such as recognizing license plates, measuring parking time, and calculating fees.

[0010] "Means for matching license plate information with a resident database" means a device, technology, or system that has the function of comparing acquired license plate information with resident registration data and determining whether the vehicle belongs to the resident.

[0011] "Means for providing stored parking records and video data in response to inquiries" refers to a device, technology, or system that has the function of searching for and providing stored parking history and related video information in response to inquiries from users or administrators.

[0012] "A means of providing free parking for up to 20 minutes and charging for every 5 minutes beyond 21 minutes" refers to a device, technology, or system that has the function of exempting parking fees for parking times of up to 20 minutes and charging a set rate for every 5 minutes beyond 21 minutes. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. The system includes means for recognizing license plates, measuring parking time, and automatically charging.

[0035] The main components of the system are:

[0036] 1. How to recognize license plates

[0037] 2. A means of recording parking start and end times

[0038] 3. Means of charging based on parking time

[0039] 4. AI-based systems that interface these methods

[0040] Overall system operation

[0041] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate. The server analyzes the image data sent and extracts the text data of the license plate. The server then checks the extracted license plate information against a resident database to confirm whether it matches. The server then records the start time of parking, and again the end time when the user's errand is finished. The server then charges based on the parking time, and saves the record.

[0042] License Plate Recognition

[0043] The device (camera) recognizes the vehicle's license plate. The camera captures the license plate image data and sends it to a server in real time. AI on the server analyzes the image data and extracts the license plate number as text data.

[0044] Parking time record

[0045] The server records the parking start time along with the license plate information. When the vehicle parked in the parking space leaves, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time using the start and end time information.

[0046] Billing process

[0047] The server calculates the parking time, charging for parking up to 20 minutes for free, and for parking over 21 minutes for every 5 minutes. This charge is automatically added to the resident's management fee.

[0048] Record keeping and inquiries

[0049] The terminal (server) stores all parking logs and video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs and video data and provide them promptly.

[0050] Specific examples

[0051] Example 1: Resident A parks his car in a temporary parking space. The terminal (camera) takes a photo of the car's license plate number "ABC-1234", and the server extracts the license plate information as text data and records the parking start time. When Resident A returns 10 minutes later, he is photographed again as he leaves the parking space, and the server records the parking end time and processes the parking as free.

[0052] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a photo of the license plate "DEF-5678", and the server calculates the parking time as 30 minutes from the information taken by taking another photo at the end of the parking period. The server calculates the charge as 200 yen for every 5 minutes from the 21st minute onwards (2 times), and adds 400 yen to the management fee.

[0053] In this way, by following this embodiment, the problem of temporary parking in apartment buildings can be solved, and both convenience and good manners can be achieved.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate.

[0057] Step 2:

[0058] The image data of the license plate captured by the terminal (camera) is sent to the server in real time.

[0059] Step 3:

[0060] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number (e.g., "ABC-1234") as text data.

[0061] Step 4:

[0062] The server temporarily stores the extracted license plate information together with time information in a database.

[0063] Step 5:

[0064] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[0065] Step 6:

[0066] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[0067] Step 7:

[0068] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures an image of the license plate.

[0069] Step 8:

[0070] The image data of the license plate captured by the terminal (camera) is sent back to the server.

[0071] Step 9:

[0072] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[0073] Step 10:

[0074] The server records the parking end time together with the license plate information in a database.

[0075] Step 11:

[0076] The server checks the parking start time and parking end time and calculates the total parking time.

[0077] Step 12:

[0078] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[0079] Step 13:

[0080] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[0081] Step 14:

[0082] The server stores all parking logs and related video data in a database along with the calculation results.

[0083] Step 15:

[0084] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[0085] Step 16:

[0086] When a user (resident) makes an inquiry, the server searches the stored parking logs and video data and provides them promptly.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] In apartment complexes, there are problems with unauthorized parking and long-term parking in temporary parking spaces reserved for residents. This problem causes inconvenience to other residents and hinders the efficient use of temporary parking spaces. Furthermore, without a system in place to properly charge based on parking time, there is a high possibility of problems arising due to a lack of fairness and transparency. For this reason, a system is needed to efficiently manage parking spaces, prevent unauthorized parking, and properly charge.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on parking time, a means for analyzing transmitted image data and extracting license plate text data, and a means for comparing the extracted license plate information with a database. This allows for real-time recognition of license plates, accurate recording of parking start and end times, and appropriate charging. It also prevents unauthorized parking and realizes efficient management of temporary parking spaces reserved for residents.

[0092] "Means for recognizing license plates" refers to a device or technology that uses a camera or other device to photograph a vehicle's license plate and identify the license plate information from the image.

[0093] The "means for recording the vehicle parking start and end times" refers to a device or technology for recording the time stamps when a vehicle enters and leaves a parking space.

[0094] The "means for charging based on parking time" refers to a device or technology for calculating parking time based on the recorded parking start time and end time, and for processing charges based on the result.

[0095] "Means for analyzing transmitted image data and extracting license plate text data" refers to a device or technology for processing image data transmitted from a camera or the like and extracting the license plate text information contained in the image in text format.

[0096] "Means for matching extracted license plate information with a database" refers to a device or technique for comparing extracted license plate text data with existing database information to search for matching information.

[0097] The "resident database" is a database for managing information about residents of an apartment complex, and specifically includes information such as license plate information for each resident.

[0098] An "artificial intelligence system" is a system that uses artificial intelligence technology to analyze and process data, specifically automating functions such as license plate recognition and data matching.

[0099] "Means for providing stored parking logs and video data in response to inquiries" refers to a device or technology for quickly searching and providing past parking records and recorded video data in response to inquiries from residents.

[0100] The present invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, preventing unauthorized parking, and automatically charging appropriate fees. This system includes means for recognizing license plates, recording parking time, and charging fees, and is described in detail below.

[0101] Hardware and software used

[0102] The main hardware and software used in this system are as follows:

[0103] Hardware

[0104] IP camera (e.g. Hikvision DS-2CD2085FWD-I)

[0105] software

[0106] OCR (Optical Character Recognition) software (e.g., Google Cloud Vision API)

[0107] AI models (e.g. custom models using TensorFlow)

[0108] Database management software (e.g., MySQL)

[0109] Program processing

[0110] The device (camera) detects vehicles entering temporary parking spaces and captures images of their license plates. The captured image data is sent to a server in real time. The server analyzes the received image data and uses OCR technology to extract the text data from the license plates. This is done using AI models such as Google Cloud Vision API and TensorFlow.

[0111] The server then compares the extracted license plate information with a database to determine whether the vehicle belongs to the resident. Database management software such as MySQL is used for the comparison. Once the verification is complete, the server records the parking start time as a timestamp.

[0112] When the user leaves the parking space after completing their errands, the device (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time by calculating the difference between the parking start time and end time.

[0113] The server processes the charges based on the calculated parking time. Parking is free for up to 20 minutes, but if it exceeds 21 minutes, a charge is made for every 5 minutes. This charge is automatically added to the resident's management fee.

[0114] Specific examples of billing processing

[0115] Example 1: Resident A parks his car in a temporary parking space. The device (camera) takes a photo of the car's license plate number "ABC-1234" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. When Resident A returns 10 minutes later and moves the car, another photo is taken, and the server records the end time of parking and processes the parking as free parking.

[0116] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a picture of license plate "DEF-5678" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. The photo is taken again 30 minutes later, and the server records the end time of parking. The server calculates the parking time as 30 minutes, charges for 10 minutes from the 21st minute onwards (200 yen x 2 times), and adds 400 yen to the management fee.

[0117] Prompt Sentence Examples

[0118] "Please explain in natural language in detail each processing step of the program's processing flow for a temporary parking management system for an apartment complex: vehicle detection and license plate recognition, converting license plate information to text and matching it with a database, recording the parking start time, recording the parking end time and calculating the parking time, charging, saving the parking log, and responding to inquiries."

[0119] In this way, by implementing the system of the present invention, the management of temporary parking spaces in apartment complexes can be made more efficient, unauthorized parking can be prevented, and appropriate charging can be achieved.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Step 1:

[0122] The terminal (camera) detects a vehicle and takes an image of the license plate. The input is a video of a vehicle entering a parking space. When the camera detects a vehicle, it captures an image of the license plate and outputs it as image data. The specific operation that takes place at this point is that the IP camera analyzes the real-time video and captures an image of the license plate when a vehicle enters the space as a trigger.

[0123] Step 2:

[0124] The input is the license plate image data captured by the camera. The terminal (camera) sends this image data to the server. The server receives the image data and proceeds to the next analysis step. The output is the image data of the license plate sent to the server. The specific operation in this step is for the terminal to convert the license plate image captured into a data packet and send it to the server via the network.

[0125] Step 3:

[0126] The input is the license plate image data received by the server. The server analyzes the received image data and extracts the text data of the license plate using OCR technology. Specifically, it uses Google Cloud Vision API and TensorFlow to extract the character information from the image. The output obtained from this process is the text data of the license plate number. The specific operation in this step is that the server inputs the image data into optical character recognition software and obtains the character information in text format.

[0127] Step 4:

[0128] The input is the extracted license plate text data. The server checks this license plate information against a resident database. This is done using a database management system such as MySQL. The output of the database check is the matching resident information, or a result if no matching information is found. The specific operation in this step is that the server queries the database for the license plate text data and searches for matching information.

[0129] Step 5:

[0130] The input is resident information obtained based on the matching. The server recognizes that the resident's vehicle has entered the parking space and records the parking start time as a timestamp. The output is the parking start time data linked to the license plate information. The specific operation in this step is for the server to obtain the current time and record it as the start time in the database.

[0131] Step 6:

[0132] When the user finishes his business and moves his vehicle out of the parking space, the terminal (camera) captures an image of the license plate again. The input is the video of the vehicle leaving the parking space. When the camera detects the vehicle leaving the parking space, it captures the image of the license plate again and outputs it as image data. Specifically, the IP camera captures the image of the license plate again, triggering the vehicle's departure.

[0133] Step 7:

[0134] The input is the re-photographed license plate image data. The terminal (camera) sends this image data to the server. The server analyzes the received image data and re-extracts the license plate text data. The output is the text data of the license plate number at the end of parking. The specific operation is to send the photographed license plate image to the server.

[0135] Step 8:

[0136] The input is the text data of the license plate at the end of parking. The server records the end time and calculates the difference from the start time to calculate the parking time. The output is the parking time data. In concrete terms, the server calculates the difference between the parking start time and the end time and records the result as the parking time.

[0137] Step 9:

[0138] The input is the calculated parking time data. The server processes the charge based on the parking time. Parking up to 20 minutes is free, and if it exceeds 21 minutes, the charge is calculated in 5-minute increments. The output is the final charge information, which is added to the management fee. The specific operation is that the server calculates the charge based on the parking time and adds that amount to the resident's account.

[0139] Step 10:

[0140] The inputs are parking start and end times, license plate information, and billing information. The server stores all parking logs and video data in a database. The output is the stored parking logs and video data. The specific operation is to store this information in the database as structured data so that it can be used to respond to future queries.

[0141] (Application example 1)

[0142] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional parking lot management systems made it difficult to accurately grasp parking lot usage, and were unable to effectively manage charges to increase customer convenience. This resulted in unauthorized parking and illegal parking, increasing store management costs. It was also difficult to respond quickly and accurately to inquiries about parking lot usage. To solve these problems, an effective parking lot management system was needed.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0145] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of parking of vehicles, a means for charging based on parking time, a means for checking the parking status in real time using a smartphone, and a means for making payments using a smartphone. This automates the management of parking lot usage, making it possible to improve the convenience of customers and the efficiency of parking lot management for stores.

[0146] A "license plate" is an identification plate attached to a vehicle that indicates the registration number.

[0147] "Means of recognition" refers to technology that uses a device such as a camera to detect a vehicle's license plate and read the information.

[0148] "Means for recording parking start and end times" refers to a system that automatically records the times a vehicle enters and leaves a parking lot.

[0149] "Charging means" refers to a system that calculates and charges fees based on parking time.

[0150] "Artificial intelligence" refers to computer programs and algorithms that automate complex processes such as data analysis and image recognition.

[0151] "Means for checking parking status in real time using a smartphone" refers to technology that allows users to grasp the current parking status in real time using a smartphone application.

[0152] "Smartphone payment method" refers to technology that allows users to pay parking fees online using a smartphone application.

[0153] A "resident database" refers to a database that stores information about residents and regular customers of apartment complexes and stores.

[0154] The present invention provides a system that recognizes vehicle license plates, records parking time, and charges at physical stores. This system has the functionality to check parking status in real time and make payments online using a smartphone.

[0155] Hardware Configuration

[0156] This system uses the following hardware:

[0157] Camera: An IP camera installed in the parking lot captures the vehicle's license plate.

[0158] Server: A server (e.g., AWS EC2) for analyzing image data, recording parking time, and processing billing.

[0159] Smartphone: A device that allows users to check parking availability in real time and make payments.

[0160] Software Configuration

[0161] This system uses the following software:

[0162] OpenCV: Image processing library. Used for license plate recognition.

[0163] pytesseract: A library for optical character recognition (OCR) to extract text data from license plates.

[0164] Flask: A web application framework that manages server-side processing.

[0165] React Native: A mobile application development framework used to develop smartphone apps.

[0166] System processing flow

[0167] 1. License Plate Recognition

[0168] The device (camera) takes a photo of the vehicle's license plate and sends the image data to the server. The AI ​​on the server uses OpenCV and pytesseract to analyze the image data and extract the text data from the license plate.

[0169] 2. Parking time record

[0170] The server records the parking start time along with the license plate information. When the vehicle leaves the parking lot, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time.

[0171] 3. Billing Process

[0172] The server processes the charges based on the calculated parking time. Parking up to 20 minutes is free, and for parking over 21 minutes, the fee is calculated in 5-minute increments. The user is then presented with a bill via a smartphone app and can pay online.

[0173] 4. Check parking status

[0174] Users can check the parking situation in real time using their smartphone, which allows them to understand the parking space usage status and remaining time.

[0175] 5. Response to inquiries

[0176] The server stores all parking logs and image data and has the functionality to respond quickly to inquiries from users, which helps prevent problems related to parking records.

[0177] Specific examples

[0178] Example 1

[0179] Customer A parks his car in the store's parking lot. The terminal (camera) takes a photo of the car's license plate, and the server extracts the license plate information as text data and records the parking start time. When Customer A leaves the parking lot 30 minutes later, the terminal (camera) takes another photo of the license plate, and the server records the parking end time. Charges are processed for the 10 minutes beyond 21 minutes, and a fee of 400 yen is displayed on the smartphone app.

[0180] Example prompts to input to the generative AI model

[0181] How can I develop an application that uses a license plate recognition system in a parking lot to automatically measure the parking time of customers and process the charges? Also, please provide the actual Python code.

[0182] In this way, by using the system of the present invention, parking lot management at physical stores can be made significantly more efficient, improving convenience for customers.

[0183] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0184] Step 1:

[0185] Photographing license plates

[0186] How it works: The device's camera detects vehicles entering a parking lot and takes an image of their license plate.

[0187] Input: Vehicle license plate number

[0188] Output: License plate image data

[0189] How it works: IP cameras installed in parking lots detect vehicles using motion detection and automatically capture images of license plates, which are then sent to a server.

[0190] Step 2:

[0191] License plate text analysis

[0192] Operation: The server analyzes the image data of the license plate received and extracts it as text data.

[0193] Input: License plate image data

[0194] Output: License plate text data

[0195] How it works: Image processing is performed on the server using OpenCV and pytesseract. First, the image is converted to grayscale, contour detection is performed, and then the license plate number is converted to text data using optical character recognition (OCR).

[0196] Step 3:

[0197] Recording parking start time

[0198] How it works: The server records the parking start time along with the license plate information in a database.

[0199] Input: License plate text data, parking start time

[0200] Output: Parking start information recorded in the database

[0201] Specific operation: Uses Python's datetime module to obtain the current time and saves it in a database along with the license plate text data.

[0202] Step 4:

[0203] Recording parking end time

[0204] Operation: When the vehicle leaves the parking lot, the device (camera) takes another image of the license plate and sends it to the server. The server analyzes the received image and records the end time.

[0205] Input: License plate image data, parking end time

[0206] Output: Parking end information recorded in the database

[0207] Specific operation: Recognize the license plate from the image using the same method as in step 2, obtain the end time and record it in the database.

[0208] Step 5:

[0209] Parking fee calculation

[0210] Operation: The server calculates the parking time based on the parking start and end times and processes the charges.

[0211] Input: Parking start time, parking end time

[0212] Output: Parking fee calculation result

[0213] Specific operation: Parking time is calculated, and parking for up to 20 minutes is free, but if it exceeds 21 minutes, 200 yen is charged for every 5 minutes. The calculated fee is sent to the user's smartphone app.

[0214] Step 6:

[0215] Real-time confirmation and payment

[0216] How it works: Users use a smartphone app to check parking availability in real time and pay online.

[0217] Input: Parking status data, parking fee data

[0218] Output: Payment completion notification

[0219] Specific operation: The user opens the smartphone app and checks the parking time and calculated fee. After the user makes the payment, the app sends a payment completion notification to the server, which records it.

[0220] Step 7:

[0221] Inquiry response

[0222] Operation: The server stores all parking logs and image data and responds to user inquiries.

[0223] Input: Parking log, image data

[0224] Output: Inquiry response results

[0225] Specific operation: Parking logs and image data are backed up regularly and can be quickly searched and provided as needed, allowing for accurate and prompt responses to user inquiries.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience, and is also combined with an emotion engine that recognizes the user's emotions. In addition to the basic functions of recognizing license plates, measuring parking time, and processing fees, this system can also analyze the user's emotions and reflect that information in the way parking fees are presented.

[0228] The main components of the system are:

[0229] 1. How to recognize license plates

[0230] 2. A means of recording parking start and end times

[0231] 3. Means of charging based on parking time

[0232] 4. AI-based systems that interface these methods

[0233] 5. Emotion engine that recognizes user emotions

[0234] Overall system operation

[0235] The terminal (camera) detects a vehicle entering a temporary parking space and takes an image of the license plate. The terminal (camera) also takes a photo of the user's facial expression, and the emotion engine analyzes the image data to recognize the user's emotions. The server receives the license plate image data and emotion analysis data and extracts the license plate number as text data. The server then compares the extracted license plate information with a resident database to confirm whether it matches, and then records the parking start time and the parking end time again when the user's errand is finished. The server then charges based on the parking time and saves the record.

[0236] License Plate Recognition and Emotion Recognition

[0237] The device (camera) recognizes the vehicle's license plate and captures the user's facial expression. The license plate image data and facial expression data captured by the camera are sent to a server in real time. The AI ​​on the server analyzes the license plate image data and extracts the license plate number as text data. At the same time, the emotion engine analyzes the received facial expression data and determines the user's emotion.

[0238] Parking time record

[0239] The server records the parking start time along with the license plate information. When the vehicle leaves the parking space, the device (camera) captures another image of the license plate and the user's facial expression and sends them to the server. The server records the license plate number and end time, and calculates the parking time and evaluates the emotion based on the parking time and emotion data.

[0240] Billing and Emotional Reflection

[0241] The server calculates the parking time, charging for parking up to 20 minutes free of charge and for parking over 21 minutes in five-minute increments. The server also adjusts the method of notifying the user of the charge (for example, by displaying a friendly message) based on the emotion data provided by the emotion engine. The charge is automatically added to the resident's management fee.

[0242] Record keeping and inquiries

[0243] The terminal (server) stores all parking logs, emotion recognition data, and related video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs, video data, and emotion recognition data and provide the information promptly.

[0244] Specific examples

[0245] Example 1: Resident A parks his / her car in a temporary parking space. The device (camera) captures the car's license plate number "ABC-1234" and Resident A's facial expression. The server extracts the license plate information as text data, and at the same time, the emotion engine recognizes the smiling expression. The parking time is 10 minutes, so it is treated as free parking, and a friendly breakdown message is displayed.

[0246] Example 2: Resident B parks for 30 minutes. The device (camera) captures the license plate number "DEF-5678" and Resident B's facial expression, and the emotion engine recognizes the stressed expression. The server calculates the parking time as 30 minutes from the information captured again at the end of parking, calculates the charge as 10 minutes from the 21st minute onwards, displays a friendly and considerate message, and adds 400 yen to the management fee.

[0247] In this way, this embodiment solves the problem of temporary parking in apartment buildings, and not only achieves both convenience and good manners, but also improves the user experience.

[0248] The processing flow will be explained below.

[0249] Step 1:

[0250] The device (camera) detects the vehicle entering and simultaneously captures the license plate and the user's facial expression.

[0251] Step 2:

[0252] The license plate image data and facial expression data captured by the terminal (camera) are sent to the server in real time.

[0253] Step 3:

[0254] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number as text data.

[0255] Step 4:

[0256] The server temporarily stores the extracted license plate information together with time information in a database.

[0257] Step 5:

[0258] The emotion engine on the server analyzes the facial expression data, recognizes the user's emotions, and records the information in a database.

[0259] Step 6:

[0260] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[0261] Step 7:

[0262] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[0263] Step 8:

[0264] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures the license plate and the user's facial expression.

[0265] Step 9:

[0266] The image data of the license plate and facial expression data captured by the terminal (camera) are sent back to the server.

[0267] Step 10:

[0268] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[0269] Step 11:

[0270] The emotion engine on the server analyzes the facial expression data again, recognizes the user's emotions, and records that information in a database.

[0271] Step 12:

[0272] The server obtains the parking start time and parking end time from the database and calculates the total parking time.

[0273] Step 13:

[0274] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[0275] Step 14:

[0276] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[0277] Step 15:

[0278] The server adjusts the method of billing notification (e.g., message content and display format) based on parking time and emotion recognition data.

[0279] Step 16:

[0280] The server stores all parking logs, emotion recognition data and related video data in a database along with the calculation results.

[0281] Step 17:

[0282] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[0283] Step 18:

[0284] When a user (resident) makes an inquiry, the server searches the stored parking logs, emotion recognition data, and video data and provides them promptly.

[0285] Example 2

[0286] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] In managing temporary parking spaces reserved for residents in apartment complexes, efficient parking management, improved user convenience, and proper parking etiquette are required. Furthermore, a method that takes into account the user's feelings is also necessary, but existing systems have difficulty achieving these simultaneously.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0289] In this invention, the server includes means for recognizing the vehicle's license plate, means for capturing the user's facial expression, means for transmitting image data of the license plate and the facial expression to the server, means for extracting the license plate information as text data, means for analyzing the user's emotions, means for recording the vehicle's parking start time and end time, means for charging based on the parking time, and means for adjusting the method of notifying the charge based on the extracted emotion information, thereby enabling efficient parking management and flexible response that takes the user's emotions into consideration.

[0290] "Means for recognizing vehicle license plates" refers to technology that uses cameras and image analysis software to detect vehicle license plates and extract that information as digital data.

[0291] The "means for capturing the user's facial expression" is a technology for capturing the user's facial expression using a camera or an image capture device and collecting the image data.

[0292] The "means for transmitting image data of license plates and facial expressions to a server" refers to the communication protocols and network technologies for transferring captured image data to a server in real time.

[0293] "Means for extracting license plate information as text data" refers to a technology that uses image analysis technology to extract the numbers and letters on license plates and convert them into text data.

[0294] "Means for analyzing user emotions" refers to technology that uses image analysis and machine learning to identify a user's emotional state from their facial expressions and obtain that information as data.

[0295] The "means for recording the start and end times of parking a vehicle" refers to a technology that automatically records the time when a vehicle enters and exits the parking lot and saves that data.

[0296] The "means for charging based on parking time" is a technology for calculating fees based on recorded parking time information and performing charging processing.

[0297] The "means for adjusting the method of billing notification based on extracted emotional information" is a technique for dynamically changing the content and display method of the notification message in consideration of the emotional state of the user.

[0298] "Means for comparing with a resident database" refers to a technology that compares the collected license plate information with a pre-registered resident information database to confirm whether the vehicle belongs to the resident.

[0299] "Means for providing parking records, video data, and emotion data" refers to technology that stores and searches past parking history, related video data, and emotion recognition results, and provides them in response to inquiries.

[0300] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, it aims to improve the user experience. The main components of this system and their specific operation are described in detail below.

[0301] System configuration

[0302] This system is constructed using the following main hardware and software:

[0303] Hardware

[0304] 1. Camera: Use a camera with high resolution and night vision, such as Sony's high-sensitivity camera.

[0305] 2. Server: Uses high-performance data processing equipment to receive, analyze, and record data.

[0306] software

[0307] 1. License plate recognition software: Detects vehicle license plates and extracts their information as text data. For example, "OpenALPR" is used.

[0308] 2. Emotion engine: Analyzes the user's facial expressions and recognizes their emotions. For example, the "Microsoft Azure Emotion Recognition API" is used.

[0309] 3. Database: A database is used to store parking logs, emotion data, and related video data.

[0310] System Operation

[0311] The device (camera) automatically detects vehicles entering temporary parking spaces and takes images of their license plates. At the same time, it also captures the facial expressions of the users inside the vehicles. This image data is sent to the server in real time.

[0312] The server first uses "OpenALPR" to extract the license plate number as text data from the received image data. At the same time, it uses an emotion engine to analyze facial expression data and recognize the user's emotions. For specific emotion recognition, it uses "Microsoft Azure Emotion Recognition API."

[0313] The server compares the extracted license plate information with the resident database, and if there is a match, it grants permission to park and records the parking start time. When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and facial expression and sends it to the server. The server records the end time, calculates the parking time, and retains the emotion data.

[0314] Charges are processed based on parking time; specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, a charge of 50 yen is applied for every 5 minutes. The method of charge notification is adjusted based on emotional data. For example, if the emotional state is recognized as stressed, a thoughtful message such as "Thank you for your hard work. Your parking time has been increased to 30 minutes. 400 yen has been added to the management fee" is displayed.

[0315] Specific examples

[0316] Example 1: Resident A

[0317] 1. The device (camera) detects the vehicle and captures the license plate number "ABC-1234" and resident A's smiling face.

[0318] 2. The server extracts the license plate number and matches it with "Resident A."

[0319] 3. The server records the parking start time.

[0320] 4. Resident A leaves 10 minutes later, and another photo is taken and the data is sent.

[0321] 5. The server records the end time and calculates the parking time.

[0322] 6. It will be treated as free parking and a friendly message will appear: "Welcome back, your parking was free for 10 minutes!"

[0323] Example 2: Resident B

[0324] 1. The device (camera) detects the vehicle and captures the license plate number "DEF-5678" and Resident B's stressed expression.

[0325] 2. The server extracts the license plate number and matches it with "Resident B."

[0326] 3. The server records the parking start time.

[0327] 4. Resident B departs 30 minutes later, and another image is taken and the data is transmitted.

[0328] 5. The server records the end time and calculates the parking time.

[0329] 6. The charge of 400 yen will be calculated and the message will be displayed: "Thank you for your time. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[0330] Prompt Sentence Examples

[0331] "I would like to create specifications for a temporary parking space management system for my apartment complex. I would like the system to include the following processes: license plate recognition, recording and charging for parking time, and analyzing user sentiment to reflect this in charging notifications. Please explain in detail the specific process flow and the technologies and tools to be used."

[0332] In this way, this system solves the problem of temporary parking in apartment complexes, achieving both convenience and good manners, as well as improving the user experience.

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] Step 1:

[0335] The device (camera) detects vehicles entering a temporary parking space and photographs the license plate and the user's facial expression. Specifically, a high-sensitivity Sony camera is used to capture the vehicle and the user's face. The input is the vehicle entering and the user's face, and the output is image data of the license plate and facial expression. This allows the initial data to be obtained.

[0336] Step 2:

[0337] The license plate and facial expression image data captured by the device (camera) is sent to the server in real time. This data transmission uses the HTTPS protocol. The input is the captured image data, and the output is the data transfer to the server. This ensures that the data is sent to the server securely.

[0338] Step 3:

[0339] The server extracts the license plate number as text data from the image data of the license plate received. Specifically, it uses license plate recognition software such as "OpenALPR." The input is the image data of the license plate, and the output is the extracted text data. This allows the specific license plate number to be obtained.

[0340] Step 4:

[0341] The server simultaneously analyzes the facial expression data using an emotion engine to recognize the user's emotions. This uses the Microsoft Azure Emotion Recognition API. The input is image data of facial expressions, and the output is the user's emotional data, which is the result of the analysis. This allows the user's emotional state to be identified.

[0342] Step 5:

[0343] The server compares the extracted license plate information with a resident database and allows parking if there is a match. This process uses a database query. The input is the text data of the license plate, and the output is the match result (allow or deny). This allows or denies parking.

[0344] Step 6:

[0345] The server records the parking start time. The time is automatically obtained from the system time. The input is the authorized license plate information, and the output is the parking start time data. This records the parking start time.

[0346] Step 7:

[0347] When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and the user's facial expression and sends them to the server. The input is the vehicle leaving the parking space and the user's face, and the output is the image data captured again. This allows the data at the end to be obtained.

[0348] Step 8:

[0349] The server records the license plate number and the end time. The input is the license plate information photographed again and the system time, and the output is the end time data. This records the end time of parking.

[0350] Step 9:

[0351] The server calculates the parking time based on the parking start time and end time. The input is the parking start time and end time, and the output is the parking time data. At the same time, it stores the emotion data provided by the emotion engine.

[0352] Step 10:

[0353] The server processes the charges based on the parking time. Specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, 50 yen is charged for every 5 minutes. The input is the parking time data, and the output is the amount charged. At this time, the notification message is adjusted based on the emotion data. For example, it displays, "Thank you for your hard work. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[0354] Step 11:

[0355] The server stores all parking logs, emotion recognition data, and related video data in a database. The input is various logs and data, and the output is stored in the database, which allows for future inquiries.

[0356] (Application example 2)

[0357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0358] Factories and logistics facilities require efficient management of temporary vehicle parking spaces while charging appropriate parking fees. It is also necessary to provide a comfortable parking experience by taking drivers' emotions into consideration. However, conventional systems have difficulty not only recognizing license plates and recording parking time, but also evaluating emotions, limiting the improvement of user experience. This creates a need for efficient parking management while reducing driver stress and dissatisfaction.

[0359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0360] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on the parking time, a means including an emotion engine for recognizing the user's emotions, and an artificial intelligence system that interfaces these means, thereby enabling efficient management of temporary vehicle parking spaces and adjustment of parking fees taking into account the driver's emotions.

[0361] A "means for recognizing license plates" is a device or system that uses a camera or image recognition technology to read a vehicle's license plate and extract that information as digital data.

[0362] The "means for recording the start and end times of vehicle parking" is a system that detects the moment a vehicle enters and exits a parking space and automatically records those times.

[0363] The "means for charging based on parking time" is a system for calculating parking fees based on the recorded parking time and charging the appropriate amount.

[0364] "Means including an emotion engine that recognizes the user's emotions" refers to a program or device that uses a camera or sensor to analyze the user's facial expressions and movements and determine their emotions.

[0365] "A system using artificial intelligence to interface these means" is a system equipped with artificial intelligence that integrates each of the above means and controls and manages them to operate as a whole.

[0366] "Means for comparing with a user database to determine whether the vehicle belongs to the user" refers to a system that compares the collected license plate information with an existing database to determine whether the vehicle belongs to a legitimate user.

[0367] The "means for providing stored parking records and video data in response to inquiries" refers to a system that searches for past parking records and video data as needed and responds to users.

[0368] "A means of charging in 5-minute increments for parking exceeding 21 minutes" refers to a system that has the function of charging for the excess parking time in fixed time increments if the parking time exceeds the designated free time.

[0369] The "means for adjusting the method of charge notification based on the user's emotions" is a system that selects an appropriate message and notification method based on the recognized emotional state of the user and notifies the user of the parking fee.

[0370] This invention is a parking space management system for designated areas within a factory, and combines license plate recognition, parking time measurement, charging processing, and an emotion engine that recognizes user emotions. This system uses various hardware and software to operate in an integrated manner.

[0371] The overall system configuration is as follows:

[0372] Hardware used:

[0373] Cameras (used to capture vehicle license plates and driver faces)

[0374] Factory robots (used to manage parking spaces)

[0375] Software used:

[0376] OpenCV (library for processing images from the camera)

[0377] EmotionEngine (software that analyzes user emotions)

[0378] PlateRecognition (license plate recognition software)

[0379] datetime (standard library for measuring parking time)

[0380] System operation overview:

[0381] The server captures the vehicle's license plate and the driver's facial expression through a camera, and recognizes the license plate number and the driver's emotions from the captured data. It converts the license plate information into text data and records the parking start and end times. Based on this, it calculates the parking time and processes the appropriate fee. It also adjusts the fee notification method based on the driver's emotions.

[0382] Examples:

[0383] Example 1: Truck A parks in a designated area within the factory. The camera recognizes the license plate number of truck "TRK-1234" and the smiling face of the driver. The parking time is 18 minutes, and the parking is treated as free parking. A friendly message is displayed.

[0384] Example 2: Truck B parks for 40 minutes. The camera recognizes the license plate number "TRK-5678" and the driver's stressed facial expression. The charging system calculates the 40 minutes at the end of parking and charges 200 yen for the 20 minutes over 21 minutes. A friendly and considerate message is displayed.

[0385] Example prompt sentence:

[0386] "Please recognize the license plate of a truck in a parking space in a designated area within the factory, measure the parking time using a rule-based AI model that analyzes the driver's facial expressions, and calculate the parking fee. Please generate a fee calculation for 45 minutes of parking and a corresponding message if the driver seems stressed."

[0387] In this way, the embodiment of the present invention enables efficient management of parking spaces within a factory and adjustment of parking fees taking into account the driver's emotions.

[0388] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0389] Step 1:

[0390] License plate and facial expression capture

[0391] Input: Video data from the device (camera)

[0392] Specific operation: The device (camera) detects a vehicle entering a parking space, captures an image of the license plate and the driver's facial expression, and transmits this video data to a server in real time.

[0393] Output: License plate image data and facial expression image data

[0394] Step 2:

[0395] License Plate Recognition

[0396] Input: License plate image data

[0397] Specific operation: The server receives the transmitted license plate image data and uses PlateRecognition software to extract the license plate number as text data.

[0398] Output: License plate number text data

[0399] Step 3:

[0400] Emotion recognition

[0401] Input: facial expression image data

[0402] Specific operation: The server receives the transmitted facial expression image data, analyzes the driver's emotions using the EmotionEngine software, and extracts the driver's emotional state as text data.

[0403] Output: Text data of the driver's emotional state

[0404] Step 4:

[0405] Recording parking start time

[0406] Input: License plate number text data and driver emotional state text data

[0407] Specific operation: The server receives the license plate number and emotional state, and records the current time as the parking start time.

[0408] Output: Recorded data of parking start time

[0409] Step 5:

[0410] Recording parking end time

[0411] Input: Video data (recapture) from the device (camera)

[0412] Specific operation: When the vehicle leaves the parking space, the device (camera) captures the license plate number and the driver's facial expression again and sends them to the server. The server recognizes the recaptured license plate number and emotional state and records the current time as the parking end time.

[0413] Output: Recorded data of parking end time

[0414] Step 6:

[0415] Parking time calculation and charging

[0416] Input: Recorded data of parking start time and parking end time

[0417] Specific operation: The server calculates the parking time based on the parking start time and parking end time, and calculates the charge amount according to the parking time. The emotion engine generates a message to adjust the charge notification method based on the driver's emotional state analyzed.

[0418] Output: Charge amount and notification message

[0419] Step 7:

[0420] Billing notifications and logging

[0421] Input: Charge amount and notification message

[0422] Specific operation: The server notifies the driver of the calculated fee and the generated notification message. It also stores all parking logs, emotion recognition data, and related video data in a database and records the logs for future inquiries.

[0423] Output: Notification to the driver and recording data to the database

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0425] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0427] [Second embodiment]

[0428] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0429] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0430] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0432] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0434] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0435] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0436] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0439] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0440] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. The system includes means for recognizing license plates, measuring parking time, and automatically charging.

[0441] The main components of the system are:

[0442] 1. How to recognize license plates

[0443] 2. A means of recording parking start and end times

[0444] 3. Means of charging based on parking time

[0445] 4. AI-based systems that interface these methods

[0446] Overall system operation

[0447] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate. The server analyzes the image data sent and extracts the text data of the license plate. The server then checks the extracted license plate information against a resident database to confirm whether it matches. The server then records the start time of parking, and again the end time when the user's errand is finished. The server then charges based on the parking time, and saves the record.

[0448] License Plate Recognition

[0449] The device (camera) recognizes the vehicle's license plate. The camera captures the license plate image data and sends it to a server in real time. AI on the server analyzes the image data and extracts the license plate number as text data.

[0450] Parking time record

[0451] The server records the parking start time along with the license plate information. When the vehicle parked in the parking space leaves, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time using the start and end time information.

[0452] Billing process

[0453] The server calculates the parking time, charging for parking up to 20 minutes for free, and for parking over 21 minutes for every 5 minutes. This charge is automatically added to the resident's management fee.

[0454] Record keeping and inquiries

[0455] The terminal (server) stores all parking logs and video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs and video data and provide them promptly.

[0456] Specific examples

[0457] Example 1: Resident A parks his car in a temporary parking space. The terminal (camera) takes a photo of the car's license plate number "ABC-1234", and the server extracts the license plate information as text data and records the parking start time. When Resident A returns 10 minutes later, he is photographed again as he leaves the parking space, and the server records the parking end time and processes the parking as free.

[0458] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a photo of the license plate "DEF-5678", and the server calculates the parking time as 30 minutes from the information taken by taking another photo at the end of the parking period. The server calculates the charge as 200 yen for every 5 minutes from the 21st minute onwards (2 times), and adds 400 yen to the management fee.

[0459] In this way, by following this embodiment, the problem of temporary parking in apartment buildings can be solved, and both convenience and good manners can be achieved.

[0460] The processing flow will be explained below.

[0461] Step 1:

[0462] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate.

[0463] Step 2:

[0464] The image data of the license plate captured by the terminal (camera) is sent to the server in real time.

[0465] Step 3:

[0466] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number (e.g., "ABC-1234") as text data.

[0467] Step 4:

[0468] The server temporarily stores the extracted license plate information together with time information in a database.

[0469] Step 5:

[0470] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[0471] Step 6:

[0472] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[0473] Step 7:

[0474] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures an image of the license plate.

[0475] Step 8:

[0476] The image data of the license plate captured by the terminal (camera) is sent back to the server.

[0477] Step 9:

[0478] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[0479] Step 10:

[0480] The server records the parking end time together with the license plate information in a database.

[0481] Step 11:

[0482] The server checks the parking start time and parking end time and calculates the total parking time.

[0483] Step 12:

[0484] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[0485] Step 13:

[0486] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[0487] Step 14:

[0488] The server stores all parking logs and related video data in a database along with the calculation results.

[0489] Step 15:

[0490] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[0491] Step 16:

[0492] When a user (resident) makes an inquiry, the server searches the stored parking logs and video data and provides them promptly.

[0493] Example 1

[0494] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0495] In apartment complexes, there are problems with unauthorized parking and long-term parking in temporary parking spaces reserved for residents. This problem causes inconvenience to other residents and hinders the efficient use of temporary parking spaces. Furthermore, without a system in place to properly charge based on parking time, there is a high possibility of problems arising due to a lack of fairness and transparency. For this reason, a system is needed to efficiently manage parking spaces, prevent unauthorized parking, and properly charge.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0497] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on parking time, a means for analyzing transmitted image data and extracting license plate text data, and a means for comparing the extracted license plate information with a database. This allows for real-time recognition of license plates, accurate recording of parking start and end times, and appropriate charging. It also prevents unauthorized parking and realizes efficient management of temporary parking spaces reserved for residents.

[0498] "Means for recognizing license plates" refers to a device or technology that uses a camera or other device to photograph a vehicle's license plate and identify the license plate information from the image.

[0499] The "means for recording the vehicle parking start and end times" refers to a device or technology for recording the time stamps when a vehicle enters and leaves a parking space.

[0500] The "means for charging based on parking time" refers to a device or technology for calculating parking time based on the recorded parking start time and end time, and for processing charges based on the result.

[0501] "Means for analyzing transmitted image data and extracting license plate text data" refers to a device or technology for processing image data transmitted from a camera or the like and extracting the license plate text information contained in the image in text format.

[0502] "Means for matching extracted license plate information with a database" refers to a device or technique for comparing extracted license plate text data with existing database information to search for matching information.

[0503] The "resident database" is a database for managing information about residents of an apartment complex, and specifically includes information such as license plate information for each resident.

[0504] An "artificial intelligence system" is a system that uses artificial intelligence technology to analyze and process data, specifically automating functions such as license plate recognition and data matching.

[0505] "Means for providing stored parking logs and video data in response to inquiries" refers to a device or technology for quickly searching and providing past parking records and recorded video data in response to inquiries from residents.

[0506] The present invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, preventing unauthorized parking, and automatically charging appropriate fees. This system includes means for recognizing license plates, recording parking time, and charging fees, and is described in detail below.

[0507] Hardware and software used

[0508] The main hardware and software used in this system are as follows:

[0509] Hardware

[0510] IP camera (e.g. Hikvision DS-2CD2085FWD-I)

[0511] software

[0512] OCR (Optical Character Recognition) software (e.g., Google Cloud Vision API)

[0513] AI models (e.g. custom models using TensorFlow)

[0514] Database management software (e.g., MySQL)

[0515] Program processing

[0516] The device (camera) detects vehicles entering temporary parking spaces and captures images of their license plates. The captured image data is sent to a server in real time. The server analyzes the received image data and uses OCR technology to extract the text data from the license plates. This is done using AI models such as Google Cloud Vision API and TensorFlow.

[0517] The server then compares the extracted license plate information with a database to determine whether the vehicle belongs to the resident. Database management software such as MySQL is used for the comparison. Once the verification is complete, the server records the parking start time as a timestamp.

[0518] When the user leaves the parking space after completing their errands, the device (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time by calculating the difference between the parking start time and end time.

[0519] The server processes the charges based on the calculated parking time. Parking is free for up to 20 minutes, but if it exceeds 21 minutes, a charge is made for every 5 minutes. This charge is automatically added to the resident's management fee.

[0520] Specific examples of billing processing

[0521] Example 1: Resident A parks his car in a temporary parking space. The device (camera) takes a photo of the car's license plate number "ABC-1234" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. When Resident A returns 10 minutes later and moves the car, another photo is taken, and the server records the end time of parking and processes the parking as free parking.

[0522] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a picture of license plate "DEF-5678" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. The photo is taken again 30 minutes later, and the server records the end time of parking. The server calculates the parking time as 30 minutes, charges for 10 minutes from the 21st minute onwards (200 yen x 2 times), and adds 400 yen to the management fee.

[0523] Prompt Sentence Examples

[0524] "Please explain in natural language in detail each processing step of the program's processing flow for a temporary parking management system for an apartment complex: vehicle detection and license plate recognition, converting license plate information to text and matching it with a database, recording the parking start time, recording the parking end time and calculating the parking time, charging, saving the parking log, and responding to inquiries."

[0525] In this way, by implementing the system of the present invention, the management of temporary parking spaces in apartment complexes can be made more efficient, unauthorized parking can be prevented, and appropriate charging can be achieved.

[0526] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0527] Step 1:

[0528] The terminal (camera) detects a vehicle and takes an image of the license plate. The input is a video of a vehicle entering a parking space. When the camera detects a vehicle, it captures an image of the license plate and outputs it as image data. The specific operation that takes place at this point is that the IP camera analyzes the real-time video and captures an image of the license plate when a vehicle enters the space as a trigger.

[0529] Step 2:

[0530] The input is the license plate image data captured by the camera. The terminal (camera) sends this image data to the server. The server receives the image data and proceeds to the next analysis step. The output is the image data of the license plate sent to the server. The specific operation in this step is for the terminal to convert the license plate image captured into a data packet and send it to the server via the network.

[0531] Step 3:

[0532] The input is the license plate image data received by the server. The server analyzes the received image data and extracts the text data of the license plate using OCR technology. Specifically, it uses Google Cloud Vision API and TensorFlow to extract the character information from the image. The output obtained from this process is the text data of the license plate number. The specific operation in this step is that the server inputs the image data into optical character recognition software and obtains the character information in text format.

[0533] Step 4:

[0534] The input is the extracted license plate text data. The server checks this license plate information against a resident database. This is done using a database management system such as MySQL. The output of the database check is the matching resident information, or a result if no matching information is found. The specific operation in this step is that the server queries the database for the license plate text data and searches for matching information.

[0535] Step 5:

[0536] The input is resident information obtained based on the matching. The server recognizes that the resident's vehicle has entered the parking space and records the parking start time as a timestamp. The output is the parking start time data linked to the license plate information. The specific operation in this step is for the server to obtain the current time and record it as the start time in the database.

[0537] Step 6:

[0538] When the user finishes his business and moves his vehicle out of the parking space, the terminal (camera) captures an image of the license plate again. The input is the video of the vehicle leaving the parking space. When the camera detects the vehicle leaving the parking space, it captures the image of the license plate again and outputs it as image data. Specifically, the IP camera captures the image of the license plate again, triggering the vehicle's departure.

[0539] Step 7:

[0540] The input is the re-photographed license plate image data. The terminal (camera) sends this image data to the server. The server analyzes the received image data and re-extracts the license plate text data. The output is the text data of the license plate number at the end of parking. The specific operation is to send the photographed license plate image to the server.

[0541] Step 8:

[0542] The input is the text data of the license plate at the end of parking. The server records the end time and calculates the difference from the start time to calculate the parking time. The output is the parking time data. In concrete terms, the server calculates the difference between the parking start time and the end time and records the result as the parking time.

[0543] Step 9:

[0544] The input is the calculated parking time data. The server processes the charge based on the parking time. Parking up to 20 minutes is free, and if it exceeds 21 minutes, the charge is calculated in 5-minute increments. The output is the final charge information, which is added to the management fee. The specific operation is that the server calculates the charge based on the parking time and adds that amount to the resident's account.

[0545] Step 10:

[0546] The inputs are parking start and end times, license plate information, and billing information. The server stores all parking logs and video data in a database. The output is the stored parking logs and video data. The specific operation is to store this information in the database as structured data so that it can be used to respond to future queries.

[0547] (Application example 1)

[0548] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0549] Conventional parking lot management systems made it difficult to accurately grasp parking lot usage, and were unable to effectively manage charges to increase customer convenience. This resulted in unauthorized parking and illegal parking, increasing store management costs. It was also difficult to respond quickly and accurately to inquiries about parking lot usage. To solve these problems, an effective parking lot management system was needed.

[0550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0551] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of parking of vehicles, a means for charging based on parking time, a means for checking the parking status in real time using a smartphone, and a means for making payments using a smartphone. This automates the management of parking lot usage, making it possible to improve the convenience of customers and the efficiency of parking lot management for stores.

[0552] A "license plate" is an identification plate attached to a vehicle that indicates the registration number.

[0553] "Means of recognition" refers to technology that uses a device such as a camera to detect a vehicle's license plate and read the information.

[0554] "Means for recording parking start and end times" refers to a system that automatically records the times a vehicle enters and leaves a parking lot.

[0555] "Charging means" refers to a system that calculates and charges fees based on parking time.

[0556] "Artificial intelligence" refers to computer programs and algorithms that automate complex processes such as data analysis and image recognition.

[0557] "Means for checking parking status in real time using a smartphone" refers to technology that allows users to grasp the current parking status in real time using a smartphone application.

[0558] "Smartphone payment method" refers to technology that allows users to pay parking fees online using a smartphone application.

[0559] A "resident database" refers to a database that stores information about residents and regular customers of apartment complexes and stores.

[0560] The present invention provides a system that recognizes vehicle license plates, records parking time, and charges at physical stores. This system has the functionality to check parking status in real time and make payments online using a smartphone.

[0561] Hardware Configuration

[0562] This system uses the following hardware:

[0563] Camera: An IP camera installed in the parking lot captures the vehicle's license plate.

[0564] Server: A server (e.g., AWS EC2) for analyzing image data, recording parking time, and processing billing.

[0565] Smartphone: A device that allows users to check parking availability in real time and make payments.

[0566] Software Configuration

[0567] This system uses the following software:

[0568] OpenCV: Image processing library. Used for license plate recognition.

[0569] pytesseract: A library for optical character recognition (OCR) to extract text data from license plates.

[0570] Flask: A web application framework that manages server-side processing.

[0571] React Native: A mobile application development framework used to develop smartphone apps.

[0572] System processing flow

[0573] 1. License Plate Recognition

[0574] The device (camera) takes a photo of the vehicle's license plate and sends the image data to the server. The AI ​​on the server uses OpenCV and pytesseract to analyze the image data and extract the text data from the license plate.

[0575] 2. Parking time record

[0576] The server records the parking start time along with the license plate information. When the vehicle leaves the parking lot, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time.

[0577] 3. Billing Process

[0578] The server processes the charges based on the calculated parking time. Parking up to 20 minutes is free, and for parking over 21 minutes, the fee is calculated in 5-minute increments. The user is then presented with a bill via a smartphone app and can pay online.

[0579] 4. Check parking status

[0580] Users can check the parking situation in real time using their smartphone, which allows them to understand the parking space usage status and remaining time.

[0581] 5. Response to inquiries

[0582] The server stores all parking logs and image data and has the functionality to respond quickly to inquiries from users, which helps prevent problems related to parking records.

[0583] Specific examples

[0584] Example 1

[0585] Customer A parks his car in the store's parking lot. The terminal (camera) takes a photo of the car's license plate, and the server extracts the license plate information as text data and records the parking start time. When Customer A leaves the parking lot 30 minutes later, the terminal (camera) takes another photo of the license plate, and the server records the parking end time. Charges are processed for the 10 minutes beyond 21 minutes, and a fee of 400 yen is displayed on the smartphone app.

[0586] Example prompts to input to the generative AI model

[0587] How can I develop an application that uses a license plate recognition system in a parking lot to automatically measure the parking time of customers and process the charges? Also, please provide the actual Python code.

[0588] In this way, by using the system of the present invention, parking lot management at physical stores can be made significantly more efficient, improving convenience for customers.

[0589] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0590] Step 1:

[0591] Photographing license plates

[0592] How it works: The device's camera detects vehicles entering a parking lot and takes an image of their license plate.

[0593] Input: Vehicle license plate number

[0594] Output: License plate image data

[0595] How it works: IP cameras installed in parking lots detect vehicles using motion detection and automatically capture images of license plates, which are then sent to a server.

[0596] Step 2:

[0597] License plate text analysis

[0598] Operation: The server analyzes the image data of the license plate received and extracts it as text data.

[0599] Input: License plate image data

[0600] Output: License plate text data

[0601] How it works: Image processing is performed on the server using OpenCV and pytesseract. First, the image is converted to grayscale, contour detection is performed, and then the license plate number is converted to text data using optical character recognition (OCR).

[0602] Step 3:

[0603] Recording parking start time

[0604] How it works: The server records the parking start time along with the license plate information in a database.

[0605] Input: License plate text data, parking start time

[0606] Output: Parking start information recorded in the database

[0607] Specific operation: Uses Python's datetime module to obtain the current time and saves it in a database along with the license plate text data.

[0608] Step 4:

[0609] Recording parking end time

[0610] Operation: When the vehicle leaves the parking lot, the device (camera) takes another image of the license plate and sends it to the server. The server analyzes the received image and records the end time.

[0611] Input: License plate image data, parking end time

[0612] Output: Parking end information recorded in the database

[0613] Specific operation: Recognize the license plate from the image using the same method as in step 2, obtain the end time and record it in the database.

[0614] Step 5:

[0615] Parking fee calculation

[0616] Operation: The server calculates the parking time based on the parking start and end times and processes the charges.

[0617] Input: Parking start time, parking end time

[0618] Output: Parking fee calculation result

[0619] Specific operation: Parking time is calculated, and parking for up to 20 minutes is free, but if it exceeds 21 minutes, 200 yen is charged for every 5 minutes. The calculated fee is sent to the user's smartphone app.

[0620] Step 6:

[0621] Real-time confirmation and payment

[0622] How it works: Users use a smartphone app to check parking availability in real time and pay online.

[0623] Input: Parking status data, parking fee data

[0624] Output: Payment completion notification

[0625] Specific operation: The user opens the smartphone app and checks the parking time and calculated fee. After the user makes the payment, the app sends a payment completion notification to the server, which records it.

[0626] Step 7:

[0627] Inquiry response

[0628] Operation: The server stores all parking logs and image data and responds to user inquiries.

[0629] Input: Parking log, image data

[0630] Output: Inquiry response results

[0631] Specific operation: Parking logs and image data are backed up regularly and can be quickly searched and provided as needed, allowing for accurate and prompt responses to user inquiries.

[0632] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0633] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience, and is also combined with an emotion engine that recognizes the user's emotions. In addition to the basic functions of recognizing license plates, measuring parking time, and processing fees, this system can also analyze the user's emotions and reflect that information in the way parking fees are presented.

[0634] The main components of the system are:

[0635] 1. How to recognize license plates

[0636] 2. A means of recording parking start and end times

[0637] 3. Means of charging based on parking time

[0638] 4. AI-based systems that interface these methods

[0639] 5. Emotion engine that recognizes user emotions

[0640] Overall system operation

[0641] The terminal (camera) detects a vehicle entering a temporary parking space and takes an image of the license plate. The terminal (camera) also takes a photo of the user's facial expression, and the emotion engine analyzes the image data to recognize the user's emotions. The server receives the license plate image data and emotion analysis data and extracts the license plate number as text data. The server then compares the extracted license plate information with a resident database to confirm whether it matches, and then records the parking start time and the parking end time again when the user's errand is finished. The server then charges based on the parking time and saves the record.

[0642] License Plate Recognition and Emotion Recognition

[0643] The device (camera) recognizes the vehicle's license plate and captures the user's facial expression. The license plate image data and facial expression data captured by the camera are sent to a server in real time. The AI ​​on the server analyzes the license plate image data and extracts the license plate number as text data. At the same time, the emotion engine analyzes the received facial expression data and determines the user's emotion.

[0644] Parking time record

[0645] The server records the parking start time along with the license plate information. When the vehicle leaves the parking space, the device (camera) captures another image of the license plate and the user's facial expression and sends them to the server. The server records the license plate number and end time, and calculates the parking time and evaluates the emotion based on the parking time and emotion data.

[0646] Billing and Emotional Reflection

[0647] The server calculates the parking time, charging for parking up to 20 minutes free of charge and for parking over 21 minutes in five-minute increments. The server also adjusts the method of notifying the user of the charge (for example, by displaying a friendly message) based on the emotion data provided by the emotion engine. The charge is automatically added to the resident's management fee.

[0648] Record keeping and inquiries

[0649] The terminal (server) stores all parking logs, emotion recognition data, and related video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs, video data, and emotion recognition data and provide the information promptly.

[0650] Specific examples

[0651] Example 1: Resident A parks his / her car in a temporary parking space. The device (camera) captures the car's license plate number "ABC-1234" and Resident A's facial expression. The server extracts the license plate information as text data, and at the same time, the emotion engine recognizes the smiling expression. The parking time is 10 minutes, so it is treated as free parking, and a friendly breakdown message is displayed.

[0652] Example 2: Resident B parks for 30 minutes. The device (camera) captures the license plate number "DEF-5678" and Resident B's facial expression, and the emotion engine recognizes the stressed expression. The server calculates the parking time as 30 minutes from the information captured again at the end of parking, calculates the charge as 10 minutes from the 21st minute onwards, displays a friendly and considerate message, and adds 400 yen to the management fee.

[0653] In this way, this embodiment solves the problem of temporary parking in apartment buildings, and not only achieves both convenience and good manners, but also improves the user experience.

[0654] The processing flow will be explained below.

[0655] Step 1:

[0656] The device (camera) detects the vehicle entering and simultaneously captures the license plate and the user's facial expression.

[0657] Step 2:

[0658] The license plate image data and facial expression data captured by the terminal (camera) are sent to the server in real time.

[0659] Step 3:

[0660] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number as text data.

[0661] Step 4:

[0662] The server temporarily stores the extracted license plate information together with time information in a database.

[0663] Step 5:

[0664] The emotion engine on the server analyzes the facial expression data, recognizes the user's emotions, and records the information in a database.

[0665] Step 6:

[0666] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[0667] Step 7:

[0668] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[0669] Step 8:

[0670] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures the license plate and the user's facial expression.

[0671] Step 9:

[0672] The image data of the license plate and facial expression data captured by the terminal (camera) are sent back to the server.

[0673] Step 10:

[0674] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[0675] Step 11:

[0676] The emotion engine on the server analyzes the facial expression data again, recognizes the user's emotions, and records that information in a database.

[0677] Step 12:

[0678] The server obtains the parking start time and parking end time from the database and calculates the total parking time.

[0679] Step 13:

[0680] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[0681] Step 14:

[0682] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[0683] Step 15:

[0684] The server adjusts the method of billing notification (e.g., message content and display format) based on parking time and emotion recognition data.

[0685] Step 16:

[0686] The server stores all parking logs, emotion recognition data and related video data in a database along with the calculation results.

[0687] Step 17:

[0688] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[0689] Step 18:

[0690] When a user (resident) makes an inquiry, the server searches the stored parking logs, emotion recognition data, and video data and provides them promptly.

[0691] Example 2

[0692] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] In managing temporary parking spaces reserved for residents in apartment complexes, efficient parking management, improved user convenience, and proper parking etiquette are required. Furthermore, a method that takes into account the user's feelings is also necessary, but existing systems have difficulty achieving these simultaneously.

[0694] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0695] In this invention, the server includes means for recognizing the vehicle's license plate, means for capturing the user's facial expression, means for transmitting image data of the license plate and the facial expression to the server, means for extracting the license plate information as text data, means for analyzing the user's emotions, means for recording the vehicle's parking start time and end time, means for charging based on the parking time, and means for adjusting the method of notifying the charge based on the extracted emotion information, thereby enabling efficient parking management and flexible response that takes the user's emotions into consideration.

[0696] "Means for recognizing vehicle license plates" refers to technology that uses cameras and image analysis software to detect vehicle license plates and extract that information as digital data.

[0697] The "means for capturing the user's facial expression" is a technology for capturing the user's facial expression using a camera or an image capture device and collecting the image data.

[0698] The "means for transmitting image data of license plates and facial expressions to a server" refers to the communication protocols and network technologies for transferring captured image data to a server in real time.

[0699] "Means for extracting license plate information as text data" refers to a technology that uses image analysis technology to extract the numbers and letters on license plates and convert them into text data.

[0700] "Means for analyzing user emotions" refers to technology that uses image analysis and machine learning to identify a user's emotional state from their facial expressions and obtain that information as data.

[0701] The "means for recording the start and end times of parking a vehicle" refers to a technology that automatically records the time when a vehicle enters and exits the parking lot and saves that data.

[0702] The "means for charging based on parking time" is a technology for calculating fees based on recorded parking time information and performing charging processing.

[0703] The "means for adjusting the method of billing notification based on extracted emotional information" is a technique for dynamically changing the content and display method of the notification message in consideration of the emotional state of the user.

[0704] "Means for comparing with a resident database" refers to a technology that compares the collected license plate information with a pre-registered resident information database to confirm whether the vehicle belongs to the resident.

[0705] "Means for providing parking records, video data, and emotion data" refers to technology that stores and searches past parking history, related video data, and emotion recognition results, and provides them in response to inquiries.

[0706] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, it aims to improve the user experience. The main components of this system and their specific operation are described in detail below.

[0707] System configuration

[0708] This system is constructed using the following main hardware and software:

[0709] Hardware

[0710] 1. Camera: Use a camera with high resolution and night vision, such as Sony's high-sensitivity camera.

[0711] 2. Server: Uses high-performance data processing equipment to receive, analyze, and record data.

[0712] software

[0713] 1. License plate recognition software: Detects vehicle license plates and extracts their information as text data. For example, "OpenALPR" is used.

[0714] 2. Emotion engine: Analyzes the user's facial expressions and recognizes their emotions. For example, the "Microsoft Azure Emotion Recognition API" is used.

[0715] 3. Database: A database is used to store parking logs, emotion data, and related video data.

[0716] System Operation

[0717] The device (camera) automatically detects vehicles entering temporary parking spaces and takes images of their license plates. At the same time, it also captures the facial expressions of the users inside the vehicles. This image data is sent to the server in real time.

[0718] The server first uses "OpenALPR" to extract the license plate number as text data from the received image data. At the same time, it uses an emotion engine to analyze facial expression data and recognize the user's emotions. For specific emotion recognition, it uses "Microsoft Azure Emotion Recognition API."

[0719] The server compares the extracted license plate information with the resident database, and if there is a match, it grants permission to park and records the parking start time. When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and facial expression and sends it to the server. The server records the end time, calculates the parking time, and retains the emotion data.

[0720] Charges are processed based on parking time; specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, a charge of 50 yen is applied for every 5 minutes. The method of charge notification is adjusted based on emotional data. For example, if the emotional state is recognized as stressed, a thoughtful message such as "Thank you for your hard work. Your parking time has been increased to 30 minutes. 400 yen has been added to the management fee" is displayed.

[0721] Specific examples

[0722] Example 1: Resident A

[0723] 1. The device (camera) detects the vehicle and captures the license plate number "ABC-1234" and resident A's smiling face.

[0724] 2. The server extracts the license plate number and matches it with "Resident A."

[0725] 3. The server records the parking start time.

[0726] 4. Resident A leaves 10 minutes later, and another photo is taken and the data is sent.

[0727] 5. The server records the end time and calculates the parking time.

[0728] 6. It will be treated as free parking and a friendly message will appear: "Welcome back, your parking was free for 10 minutes!"

[0729] Example 2: Resident B

[0730] 1. The device (camera) detects the vehicle and captures the license plate number "DEF-5678" and Resident B's stressed expression.

[0731] 2. The server extracts the license plate number and matches it with "Resident B."

[0732] 3. The server records the parking start time.

[0733] 4. Resident B departs 30 minutes later, and another image is taken and the data is transmitted.

[0734] 5. The server records the end time and calculates the parking time.

[0735] 6. The charge of 400 yen will be calculated and the message will be displayed: "Thank you for your time. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[0736] Prompt Sentence Examples

[0737] "I would like to create specifications for a temporary parking space management system for my apartment complex. I would like the system to include the following processes: license plate recognition, recording and charging for parking time, and analyzing user sentiment to reflect this in charging notifications. Please explain in detail the specific process flow and the technologies and tools to be used."

[0738] In this way, this system solves the problem of temporary parking in apartment complexes, achieving both convenience and good manners, as well as improving the user experience.

[0739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0740] Step 1:

[0741] The device (camera) detects vehicles entering a temporary parking space and photographs the license plate and the user's facial expression. Specifically, a high-sensitivity Sony camera is used to capture the vehicle and the user's face. The input is the vehicle entering and the user's face, and the output is image data of the license plate and facial expression. This allows the initial data to be obtained.

[0742] Step 2:

[0743] The license plate and facial expression image data captured by the device (camera) is sent to the server in real time. This data transmission uses the HTTPS protocol. The input is the captured image data, and the output is the data transfer to the server. This ensures that the data is sent to the server securely.

[0744] Step 3:

[0745] The server extracts the license plate number as text data from the image data of the license plate received. Specifically, it uses license plate recognition software such as "OpenALPR." The input is the image data of the license plate, and the output is the extracted text data. This allows the specific license plate number to be obtained.

[0746] Step 4:

[0747] The server simultaneously analyzes the facial expression data using an emotion engine to recognize the user's emotions. This uses the Microsoft Azure Emotion Recognition API. The input is image data of facial expressions, and the output is the user's emotional data, which is the result of the analysis. This allows the user's emotional state to be identified.

[0748] Step 5:

[0749] The server compares the extracted license plate information with a resident database and allows parking if there is a match. This process uses a database query. The input is the text data of the license plate, and the output is the match result (allow or deny). This allows or denies parking.

[0750] Step 6:

[0751] The server records the parking start time. The time is automatically obtained from the system time. The input is the authorized license plate information, and the output is the parking start time data. This records the parking start time.

[0752] Step 7:

[0753] When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and the user's facial expression and sends them to the server. The input is the vehicle leaving the parking space and the user's face, and the output is the image data captured again. This allows the data at the end to be obtained.

[0754] Step 8:

[0755] The server records the license plate number and the end time. The input is the license plate information photographed again and the system time, and the output is the end time data. This records the end time of parking.

[0756] Step 9:

[0757] The server calculates the parking time based on the parking start time and end time. The input is the parking start time and end time, and the output is the parking time data. At the same time, it stores the emotion data provided by the emotion engine.

[0758] Step 10:

[0759] The server processes the charges based on the parking time. Specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, 50 yen is charged for every 5 minutes. The input is the parking time data, and the output is the amount charged. At this time, the notification message is adjusted based on the emotion data. For example, it displays, "Thank you for your hard work. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[0760] Step 11:

[0761] The server stores all parking logs, emotion recognition data, and related video data in a database. The input is various logs and data, and the output is stored in the database, which allows for future inquiries.

[0762] (Application example 2)

[0763] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0764] Factories and logistics facilities require efficient management of temporary vehicle parking spaces while charging appropriate parking fees. It is also necessary to provide a comfortable parking experience by taking drivers' emotions into consideration. However, conventional systems have difficulty not only recognizing license plates and recording parking time, but also evaluating emotions, limiting the improvement of user experience. This creates a need for efficient parking management while reducing driver stress and dissatisfaction.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0766] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on the parking time, a means including an emotion engine for recognizing the user's emotions, and an artificial intelligence system that interfaces these means, thereby enabling efficient management of temporary vehicle parking spaces and adjustment of parking fees taking into account the driver's emotions.

[0767] A "means for recognizing license plates" is a device or system that uses a camera or image recognition technology to read a vehicle's license plate and extract that information as digital data.

[0768] The "means for recording the start and end times of vehicle parking" is a system that detects the moment a vehicle enters and exits a parking space and automatically records those times.

[0769] The "means for charging based on parking time" is a system for calculating parking fees based on the recorded parking time and charging the appropriate amount.

[0770] "Means including an emotion engine that recognizes the user's emotions" refers to a program or device that uses a camera or sensor to analyze the user's facial expressions and movements and determine their emotions.

[0771] "A system using artificial intelligence to interface these means" is a system equipped with artificial intelligence that integrates each of the above means and controls and manages them to operate as a whole.

[0772] "Means for comparing with a user database to determine whether the vehicle belongs to the user" refers to a system that compares the collected license plate information with an existing database to determine whether the vehicle belongs to a legitimate user.

[0773] The "means for providing stored parking records and video data in response to inquiries" refers to a system that searches for past parking records and video data as needed and responds to users.

[0774] "A means of charging in 5-minute increments for parking exceeding 21 minutes" refers to a system that has the function of charging for the excess parking time in fixed time increments if the parking time exceeds the designated free time.

[0775] The "means for adjusting the method of charge notification based on the user's emotions" is a system that selects an appropriate message and notification method based on the recognized emotional state of the user and notifies the user of the parking fee.

[0776] This invention is a parking space management system for designated areas within a factory, and combines license plate recognition, parking time measurement, charging processing, and an emotion engine that recognizes user emotions. This system uses various hardware and software to operate in an integrated manner.

[0777] The overall system configuration is as follows:

[0778] Hardware used:

[0779] Cameras (used to capture vehicle license plates and driver faces)

[0780] Factory robots (used to manage parking spaces)

[0781] Software used:

[0782] OpenCV (library for processing images from the camera)

[0783] EmotionEngine (software that analyzes user emotions)

[0784] PlateRecognition (license plate recognition software)

[0785] datetime (standard library for measuring parking time)

[0786] System operation overview:

[0787] The server captures the vehicle's license plate and the driver's facial expression through a camera, and recognizes the license plate number and the driver's emotions from the captured data. It converts the license plate information into text data and records the parking start and end times. Based on this, it calculates the parking time and processes the appropriate fee. It also adjusts the fee notification method based on the driver's emotions.

[0788] Examples:

[0789] Example 1: Truck A parks in a designated area within the factory. The camera recognizes the license plate number of truck "TRK-1234" and the smiling face of the driver. The parking time is 18 minutes, and the parking is treated as free parking. A friendly message is displayed.

[0790] Example 2: Truck B parks for 40 minutes. The camera recognizes the license plate number "TRK-5678" and the driver's stressed facial expression. The charging system calculates the 40 minutes at the end of parking and charges 200 yen for the 20 minutes over 21 minutes. A friendly and considerate message is displayed.

[0791] Example prompt sentence:

[0792] "Please recognize the license plate of a truck in a parking space in a designated area within the factory, measure the parking time using a rule-based AI model that analyzes the driver's facial expressions, and calculate the parking fee. Please generate a fee calculation for 45 minutes of parking and a corresponding message if the driver seems stressed."

[0793] In this way, the embodiment of the present invention enables efficient management of parking spaces within a factory and adjustment of parking fees taking into account the driver's emotions.

[0794] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0795] Step 1:

[0796] License plate and facial expression capture

[0797] Input: Video data from the device (camera)

[0798] Specific operation: The device (camera) detects a vehicle entering a parking space, captures an image of the license plate and the driver's facial expression, and transmits this video data to a server in real time.

[0799] Output: License plate image data and facial expression image data

[0800] Step 2:

[0801] License Plate Recognition

[0802] Input: License plate image data

[0803] Specific operation: The server receives the transmitted license plate image data and uses PlateRecognition software to extract the license plate number as text data.

[0804] Output: License plate number text data

[0805] Step 3:

[0806] Emotion recognition

[0807] Input: facial expression image data

[0808] Specific operation: The server receives the transmitted facial expression image data, analyzes the driver's emotions using the EmotionEngine software, and extracts the driver's emotional state as text data.

[0809] Output: Text data of the driver's emotional state

[0810] Step 4:

[0811] Recording parking start time

[0812] Input: License plate number text data and driver emotional state text data

[0813] Specific operation: The server receives the license plate number and emotional state, and records the current time as the parking start time.

[0814] Output: Recorded data of parking start time

[0815] Step 5:

[0816] Recording parking end time

[0817] Input: Video data (recapture) from the device (camera)

[0818] Specific operation: When the vehicle leaves the parking space, the device (camera) captures the license plate number and the driver's facial expression again and sends them to the server. The server recognizes the recaptured license plate number and emotional state and records the current time as the parking end time.

[0819] Output: Recorded data of parking end time

[0820] Step 6:

[0821] Parking time calculation and charging

[0822] Input: Recorded data of parking start time and parking end time

[0823] Specific operation: The server calculates the parking time based on the parking start time and parking end time, and calculates the charge amount according to the parking time. The emotion engine generates a message to adjust the charge notification method based on the driver's emotional state analyzed.

[0824] Output: Charge amount and notification message

[0825] Step 7:

[0826] Billing notifications and logging

[0827] Input: Charge amount and notification message

[0828] Specific operation: The server notifies the driver of the calculated fee and the generated notification message. It also stores all parking logs, emotion recognition data, and related video data in a database and records the logs for future inquiries.

[0829] Output: Notification to the driver and recording data to the database

[0830] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0831] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0832] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0833] [Third embodiment]

[0834] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0835] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0836] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0837] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0838] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0839] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0840] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0841] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0842] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0843] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0844] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0845] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0846] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. The system includes means for recognizing license plates, measuring parking time, and automatically charging.

[0847] The main components of the system are:

[0848] 1. How to recognize license plates

[0849] 2. A means of recording parking start and end times

[0850] 3. Means of charging based on parking time

[0851] 4. AI-based systems that interface these methods

[0852] Overall system operation

[0853] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate. The server analyzes the image data sent and extracts the text data of the license plate. The server then checks the extracted license plate information against a resident database to confirm whether it matches. The server then records the start time of parking, and again the end time when the user's errand is finished. The server then charges based on the parking time, and saves the record.

[0854] License Plate Recognition

[0855] The device (camera) recognizes the vehicle's license plate. The camera captures the license plate image data and sends it to a server in real time. AI on the server analyzes the image data and extracts the license plate number as text data.

[0856] Parking time record

[0857] The server records the parking start time along with the license plate information. When the vehicle parked in the parking space leaves, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time using the start and end time information.

[0858] Billing process

[0859] The server calculates the parking time, charging for parking up to 20 minutes for free, and for parking over 21 minutes for every 5 minutes. This charge is automatically added to the resident's management fee.

[0860] Record keeping and inquiries

[0861] The terminal (server) stores all parking logs and video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs and video data and provide them promptly.

[0862] Specific examples

[0863] Example 1: Resident A parks his car in a temporary parking space. The terminal (camera) takes a photo of the car's license plate number "ABC-1234", and the server extracts the license plate information as text data and records the parking start time. When Resident A returns 10 minutes later, he is photographed again as he leaves the parking space, and the server records the parking end time and processes the parking as free.

[0864] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a photo of the license plate "DEF-5678", and the server calculates the parking time as 30 minutes from the information taken by taking another photo at the end of the parking period. The server calculates the charge as 200 yen for every 5 minutes from the 21st minute onwards (2 times), and adds 400 yen to the management fee.

[0865] In this way, by following this embodiment, the problem of temporary parking in apartment buildings can be solved, and both convenience and good manners can be achieved.

[0866] The processing flow will be explained below.

[0867] Step 1:

[0868] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate.

[0869] Step 2:

[0870] The image data of the license plate captured by the terminal (camera) is sent to the server in real time.

[0871] Step 3:

[0872] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number (e.g., "ABC-1234") as text data.

[0873] Step 4:

[0874] The server temporarily stores the extracted license plate information together with time information in a database.

[0875] Step 5:

[0876] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[0877] Step 6:

[0878] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[0879] Step 7:

[0880] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures an image of the license plate.

[0881] Step 8:

[0882] The image data of the license plate captured by the terminal (camera) is sent back to the server.

[0883] Step 9:

[0884] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[0885] Step 10:

[0886] The server records the parking end time together with the license plate information in a database.

[0887] Step 11:

[0888] The server checks the parking start time and parking end time and calculates the total parking time.

[0889] Step 12:

[0890] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[0891] Step 13:

[0892] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[0893] Step 14:

[0894] The server stores all parking logs and related video data in a database along with the calculation results.

[0895] Step 15:

[0896] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[0897] Step 16:

[0898] When a user (resident) makes an inquiry, the server searches the stored parking logs and video data and provides them promptly.

[0899] Example 1

[0900] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0901] In apartment complexes, there are problems with unauthorized parking and long-term parking in temporary parking spaces reserved for residents. This problem causes inconvenience to other residents and hinders the efficient use of temporary parking spaces. Furthermore, without a system in place to properly charge based on parking time, there is a high possibility of problems arising due to a lack of fairness and transparency. For this reason, a system is needed to efficiently manage parking spaces, prevent unauthorized parking, and properly charge.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0903] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on parking time, a means for analyzing transmitted image data and extracting license plate text data, and a means for comparing the extracted license plate information with a database. This allows for real-time recognition of license plates, accurate recording of parking start and end times, and appropriate charging. It also prevents unauthorized parking and realizes efficient management of temporary parking spaces reserved for residents.

[0904] "Means for recognizing license plates" refers to a device or technology that uses a camera or other device to photograph a vehicle's license plate and identify the license plate information from the image.

[0905] The "means for recording the vehicle parking start and end times" refers to a device or technology for recording the time stamps when a vehicle enters and leaves a parking space.

[0906] The "means for charging based on parking time" refers to a device or technology for calculating parking time based on the recorded parking start time and end time, and for processing charges based on the result.

[0907] "Means for analyzing transmitted image data and extracting license plate text data" refers to a device or technology for processing image data transmitted from a camera or the like and extracting the license plate text information contained in the image in text format.

[0908] "Means for matching extracted license plate information with a database" refers to a device or technique for comparing extracted license plate text data with existing database information to search for matching information.

[0909] The "resident database" is a database for managing information about residents of an apartment complex, and specifically includes information such as license plate information for each resident.

[0910] An "artificial intelligence system" is a system that uses artificial intelligence technology to analyze and process data, specifically automating functions such as license plate recognition and data matching.

[0911] "Means for providing stored parking logs and video data in response to inquiries" refers to a device or technology for quickly searching and providing past parking records and recorded video data in response to inquiries from residents.

[0912] The present invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, preventing unauthorized parking, and automatically charging appropriate fees. This system includes means for recognizing license plates, recording parking time, and charging fees, and is described in detail below.

[0913] Hardware and software used

[0914] The main hardware and software used in this system are as follows:

[0915] Hardware

[0916] IP camera (e.g. Hikvision DS-2CD2085FWD-I)

[0917] software

[0918] OCR (Optical Character Recognition) software (e.g., Google Cloud Vision API)

[0919] AI models (e.g. custom models using TensorFlow)

[0920] Database management software (e.g., MySQL)

[0921] Program processing

[0922] The device (camera) detects vehicles entering temporary parking spaces and captures images of their license plates. The captured image data is sent to a server in real time. The server analyzes the received image data and uses OCR technology to extract the text data from the license plates. This is done using AI models such as Google Cloud Vision API and TensorFlow.

[0923] The server then compares the extracted license plate information with a database to determine whether the vehicle belongs to the resident. Database management software such as MySQL is used for the comparison. Once the verification is complete, the server records the parking start time as a timestamp.

[0924] When the user leaves the parking space after completing their errands, the device (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time by calculating the difference between the parking start time and end time.

[0925] The server processes the charges based on the calculated parking time. Parking is free for up to 20 minutes, but if it exceeds 21 minutes, a charge is made for every 5 minutes. This charge is automatically added to the resident's management fee.

[0926] Specific examples of billing processing

[0927] Example 1: Resident A parks his car in a temporary parking space. The device (camera) takes a photo of the car's license plate number "ABC-1234" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. When Resident A returns 10 minutes later and moves the car, another photo is taken, and the server records the end time of parking and processes the parking as free parking.

[0928] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a picture of license plate "DEF-5678" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. The photo is taken again 30 minutes later, and the server records the end time of parking. The server calculates the parking time as 30 minutes, charges for 10 minutes from the 21st minute onwards (200 yen x 2 times), and adds 400 yen to the management fee.

[0929] Prompt Sentence Examples

[0930] "Please explain in natural language in detail each processing step of the program's processing flow for a temporary parking management system for an apartment complex: vehicle detection and license plate recognition, converting license plate information to text and matching it with a database, recording the parking start time, recording the parking end time and calculating the parking time, charging, saving the parking log, and responding to inquiries."

[0931] In this way, by implementing the system of the present invention, the management of temporary parking spaces in apartment complexes can be made more efficient, unauthorized parking can be prevented, and appropriate charging can be achieved.

[0932] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0933] Step 1:

[0934] The terminal (camera) detects a vehicle and takes an image of the license plate. The input is a video of a vehicle entering a parking space. When the camera detects a vehicle, it captures an image of the license plate and outputs it as image data. The specific operation that takes place at this point is that the IP camera analyzes the real-time video and captures an image of the license plate when a vehicle enters the space as a trigger.

[0935] Step 2:

[0936] The input is the license plate image data captured by the camera. The terminal (camera) sends this image data to the server. The server receives the image data and proceeds to the next analysis step. The output is the image data of the license plate sent to the server. The specific operation in this step is for the terminal to convert the license plate image captured into a data packet and send it to the server via the network.

[0937] Step 3:

[0938] The input is the license plate image data received by the server. The server analyzes the received image data and extracts the text data of the license plate using OCR technology. Specifically, it uses Google Cloud Vision API and TensorFlow to extract the character information from the image. The output obtained from this process is the text data of the license plate number. The specific operation in this step is that the server inputs the image data into optical character recognition software and obtains the character information in text format.

[0939] Step 4:

[0940] The input is the extracted license plate text data. The server checks this license plate information against a resident database. This is done using a database management system such as MySQL. The output of the database check is the matching resident information, or a result if no matching information is found. The specific operation in this step is that the server queries the database for the license plate text data and searches for matching information.

[0941] Step 5:

[0942] The input is resident information obtained based on the matching. The server recognizes that the resident's vehicle has entered the parking space and records the parking start time as a timestamp. The output is the parking start time data linked to the license plate information. The specific operation in this step is for the server to obtain the current time and record it as the start time in the database.

[0943] Step 6:

[0944] When the user finishes his business and moves his vehicle out of the parking space, the terminal (camera) captures an image of the license plate again. The input is the video of the vehicle leaving the parking space. When the camera detects the vehicle leaving the parking space, it captures the image of the license plate again and outputs it as image data. Specifically, the IP camera captures the image of the license plate again, triggering the vehicle's departure.

[0945] Step 7:

[0946] The input is the re-photographed license plate image data. The terminal (camera) sends this image data to the server. The server analyzes the received image data and re-extracts the license plate text data. The output is the text data of the license plate number at the end of parking. The specific operation is to send the photographed license plate image to the server.

[0947] Step 8:

[0948] The input is the text data of the license plate at the end of parking. The server records the end time and calculates the difference from the start time to calculate the parking time. The output is the parking time data. In concrete terms, the server calculates the difference between the parking start time and the end time and records the result as the parking time.

[0949] Step 9:

[0950] The input is the calculated parking time data. The server processes the charge based on the parking time. Parking up to 20 minutes is free, and if it exceeds 21 minutes, the charge is calculated in 5-minute increments. The output is the final charge information, which is added to the management fee. The specific operation is that the server calculates the charge based on the parking time and adds that amount to the resident's account.

[0951] Step 10:

[0952] The inputs are parking start and end times, license plate information, and billing information. The server stores all parking logs and video data in a database. The output is the stored parking logs and video data. The specific operation is to store this information in the database as structured data so that it can be used to respond to future queries.

[0953] (Application example 1)

[0954] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] Conventional parking lot management systems made it difficult to accurately grasp parking lot usage, and were unable to effectively manage charges to increase customer convenience. This resulted in unauthorized parking and illegal parking, increasing store management costs. It was also difficult to respond quickly and accurately to inquiries about parking lot usage. To solve these problems, an effective parking lot management system was needed.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0957] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of parking of vehicles, a means for charging based on parking time, a means for checking the parking status in real time using a smartphone, and a means for making payments using a smartphone. This automates the management of parking lot usage, making it possible to improve the convenience of customers and the efficiency of parking lot management for stores.

[0958] A "license plate" is an identification plate attached to a vehicle that indicates the registration number.

[0959] "Means of recognition" refers to technology that uses a device such as a camera to detect a vehicle's license plate and read the information.

[0960] "Means for recording parking start and end times" refers to a system that automatically records the times a vehicle enters and leaves a parking lot.

[0961] "Charging means" refers to a system that calculates and charges fees based on parking time.

[0962] "Artificial intelligence" refers to computer programs and algorithms that automate complex processes such as data analysis and image recognition.

[0963] "Means for checking parking status in real time using a smartphone" refers to technology that allows users to grasp the current parking status in real time using a smartphone application.

[0964] "Smartphone payment method" refers to technology that allows users to pay parking fees online using a smartphone application.

[0965] A "resident database" refers to a database that stores information about residents and regular customers of apartment complexes and stores.

[0966] The present invention provides a system that recognizes vehicle license plates, records parking time, and charges at physical stores. This system has the functionality to check parking status in real time and make payments online using a smartphone.

[0967] Hardware Configuration

[0968] This system uses the following hardware:

[0969] Camera: An IP camera installed in the parking lot captures the vehicle's license plate.

[0970] Server: A server (e.g., AWS EC2) for analyzing image data, recording parking time, and processing billing.

[0971] Smartphone: A device that allows users to check parking availability in real time and make payments.

[0972] Software Configuration

[0973] This system uses the following software:

[0974] OpenCV: Image processing library. Used for license plate recognition.

[0975] pytesseract: A library for optical character recognition (OCR) to extract text data from license plates.

[0976] Flask: A web application framework that manages server-side processing.

[0977] React Native: A mobile application development framework used to develop smartphone apps.

[0978] System processing flow

[0979] 1. License Plate Recognition

[0980] The device (camera) takes a photo of the vehicle's license plate and sends the image data to the server. The AI ​​on the server uses OpenCV and pytesseract to analyze the image data and extract the text data from the license plate.

[0981] 2. Parking time record

[0982] The server records the parking start time along with the license plate information. When the vehicle leaves the parking lot, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time.

[0983] 3. Billing Process

[0984] The server processes the charges based on the calculated parking time. Parking up to 20 minutes is free, and for parking over 21 minutes, the fee is calculated in 5-minute increments. The user is then presented with a bill via a smartphone app and can pay online.

[0985] 4. Check parking status

[0986] Users can check the parking situation in real time using their smartphone, which allows them to understand the parking space usage status and remaining time.

[0987] 5. Response to inquiries

[0988] The server stores all parking logs and image data and has the functionality to respond quickly to inquiries from users, which helps prevent problems related to parking records.

[0989] Specific examples

[0990] Example 1

[0991] Customer A parks his car in the store's parking lot. The terminal (camera) takes a photo of the car's license plate, and the server extracts the license plate information as text data and records the parking start time. When Customer A leaves the parking lot 30 minutes later, the terminal (camera) takes another photo of the license plate, and the server records the parking end time. Charges are processed for the 10 minutes beyond 21 minutes, and a fee of 400 yen is displayed on the smartphone app.

[0992] Example prompts to input to the generative AI model

[0993] How can I develop an application that uses a license plate recognition system in a parking lot to automatically measure the parking time of customers and process the charges? Also, please provide the actual Python code.

[0994] In this way, by using the system of the present invention, parking lot management at physical stores can be made significantly more efficient, improving convenience for customers.

[0995] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0996] Step 1:

[0997] Photographing license plates

[0998] How it works: The device's camera detects vehicles entering a parking lot and takes an image of their license plate.

[0999] Input: Vehicle license plate number

[1000] Output: License plate image data

[1001] How it works: IP cameras installed in parking lots detect vehicles using motion detection and automatically capture images of license plates, which are then sent to a server.

[1002] Step 2:

[1003] License plate text analysis

[1004] Operation: The server analyzes the image data of the license plate received and extracts it as text data.

[1005] Input: License plate image data

[1006] Output: License plate text data

[1007] How it works: Image processing is performed on the server using OpenCV and pytesseract. First, the image is converted to grayscale, contour detection is performed, and then the license plate number is converted to text data using optical character recognition (OCR).

[1008] Step 3:

[1009] Recording parking start time

[1010] How it works: The server records the parking start time along with the license plate information in a database.

[1011] Input: License plate text data, parking start time

[1012] Output: Parking start information recorded in the database

[1013] Specific operation: Uses Python's datetime module to obtain the current time and saves it in a database along with the license plate text data.

[1014] Step 4:

[1015] Recording parking end time

[1016] Operation: When the vehicle leaves the parking lot, the device (camera) takes another image of the license plate and sends it to the server. The server analyzes the received image and records the end time.

[1017] Input: License plate image data, parking end time

[1018] Output: Parking end information recorded in the database

[1019] Specific operation: Recognize the license plate from the image using the same method as in step 2, obtain the end time and record it in the database.

[1020] Step 5:

[1021] Parking fee calculation

[1022] Operation: The server calculates the parking time based on the parking start and end times and processes the charges.

[1023] Input: Parking start time, parking end time

[1024] Output: Parking fee calculation result

[1025] Specific operation: Parking time is calculated, and parking for up to 20 minutes is free, but if it exceeds 21 minutes, 200 yen is charged for every 5 minutes. The calculated fee is sent to the user's smartphone app.

[1026] Step 6:

[1027] Real-time confirmation and payment

[1028] How it works: Users use a smartphone app to check parking availability in real time and pay online.

[1029] Input: Parking status data, parking fee data

[1030] Output: Payment completion notification

[1031] Specific operation: The user opens the smartphone app and checks the parking time and calculated fee. After the user makes the payment, the app sends a payment completion notification to the server, which records it.

[1032] Step 7:

[1033] Inquiry response

[1034] Operation: The server stores all parking logs and image data and responds to user inquiries.

[1035] Input: Parking log, image data

[1036] Output: Inquiry response results

[1037] Specific operation: Parking logs and image data are backed up regularly and can be quickly searched and provided as needed, allowing for accurate and prompt responses to user inquiries.

[1038] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1039] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience, and is also combined with an emotion engine that recognizes the user's emotions. In addition to the basic functions of recognizing license plates, measuring parking time, and processing fees, this system can also analyze the user's emotions and reflect that information in the way parking fees are presented.

[1040] The main components of the system are:

[1041] 1. How to recognize license plates

[1042] 2. A means of recording parking start and end times

[1043] 3. Means of charging based on parking time

[1044] 4. AI-based systems that interface these methods

[1045] 5. Emotion engine that recognizes user emotions

[1046] Overall system operation

[1047] The terminal (camera) detects a vehicle entering a temporary parking space and takes an image of the license plate. The terminal (camera) also takes a photo of the user's facial expression, and the emotion engine analyzes the image data to recognize the user's emotions. The server receives the license plate image data and emotion analysis data and extracts the license plate number as text data. The server then compares the extracted license plate information with a resident database to confirm whether it matches, and then records the parking start time and the parking end time again when the user's errand is finished. The server then charges based on the parking time and saves the record.

[1048] License Plate Recognition and Emotion Recognition

[1049] The device (camera) recognizes the vehicle's license plate and captures the user's facial expression. The license plate image data and facial expression data captured by the camera are sent to a server in real time. The AI ​​on the server analyzes the license plate image data and extracts the license plate number as text data. At the same time, the emotion engine analyzes the received facial expression data and determines the user's emotion.

[1050] Parking time record

[1051] The server records the parking start time along with the license plate information. When the vehicle leaves the parking space, the device (camera) captures another image of the license plate and the user's facial expression and sends them to the server. The server records the license plate number and end time, and calculates the parking time and evaluates the emotion based on the parking time and emotion data.

[1052] Billing and Emotional Reflection

[1053] The server calculates the parking time, charging for parking up to 20 minutes free of charge and for parking over 21 minutes in five-minute increments. The server also adjusts the method of notifying the user of the charge (for example, by displaying a friendly message) based on the emotion data provided by the emotion engine. The charge is automatically added to the resident's management fee.

[1054] Record keeping and inquiries

[1055] The terminal (server) stores all parking logs, emotion recognition data, and related video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs, video data, and emotion recognition data and provide the information promptly.

[1056] Specific examples

[1057] Example 1: Resident A parks his / her car in a temporary parking space. The device (camera) captures the car's license plate number "ABC-1234" and Resident A's facial expression. The server extracts the license plate information as text data, and at the same time, the emotion engine recognizes the smiling expression. The parking time is 10 minutes, so it is treated as free parking, and a friendly breakdown message is displayed.

[1058] Example 2: Resident B parks for 30 minutes. The device (camera) captures the license plate number "DEF-5678" and Resident B's facial expression, and the emotion engine recognizes the stressed expression. The server calculates the parking time as 30 minutes from the information captured again at the end of parking, calculates the charge as 10 minutes from the 21st minute onwards, displays a friendly and considerate message, and adds 400 yen to the management fee.

[1059] In this way, this embodiment solves the problem of temporary parking in apartment buildings, and not only achieves both convenience and good manners, but also improves the user experience.

[1060] The processing flow will be explained below.

[1061] Step 1:

[1062] The device (camera) detects the vehicle entering and simultaneously captures the license plate and the user's facial expression.

[1063] Step 2:

[1064] The license plate image data and facial expression data captured by the terminal (camera) are sent to the server in real time.

[1065] Step 3:

[1066] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number as text data.

[1067] Step 4:

[1068] The server temporarily stores the extracted license plate information together with time information in a database.

[1069] Step 5:

[1070] The emotion engine on the server analyzes the facial expression data, recognizes the user's emotions, and records the information in a database.

[1071] Step 6:

[1072] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[1073] Step 7:

[1074] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[1075] Step 8:

[1076] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures the license plate and the user's facial expression.

[1077] Step 9:

[1078] The image data of the license plate and facial expression data captured by the terminal (camera) are sent back to the server.

[1079] Step 10:

[1080] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[1081] Step 11:

[1082] The emotion engine on the server analyzes the facial expression data again, recognizes the user's emotions, and records that information in a database.

[1083] Step 12:

[1084] The server obtains the parking start time and parking end time from the database and calculates the total parking time.

[1085] Step 13:

[1086] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[1087] Step 14:

[1088] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[1089] Step 15:

[1090] The server adjusts the method of billing notification (e.g., message content and display format) based on parking time and emotion recognition data.

[1091] Step 16:

[1092] The server stores all parking logs, emotion recognition data and related video data in a database along with the calculation results.

[1093] Step 17:

[1094] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[1095] Step 18:

[1096] When a user (resident) makes an inquiry, the server searches the stored parking logs, emotion recognition data, and video data and provides them promptly.

[1097] Example 2

[1098] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1099] In managing temporary parking spaces reserved for residents in apartment complexes, efficient parking management, improved user convenience, and proper parking etiquette are required. Furthermore, a method that takes into account the user's feelings is also necessary, but existing systems have difficulty achieving these simultaneously.

[1100] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1101] In this invention, the server includes means for recognizing the vehicle's license plate, means for capturing the user's facial expression, means for transmitting image data of the license plate and the facial expression to the server, means for extracting the license plate information as text data, means for analyzing the user's emotions, means for recording the vehicle's parking start time and end time, means for charging based on the parking time, and means for adjusting the method of notifying the charge based on the extracted emotion information, thereby enabling efficient parking management and flexible response that takes the user's emotions into consideration.

[1102] "Means for recognizing vehicle license plates" refers to technology that uses cameras and image analysis software to detect vehicle license plates and extract that information as digital data.

[1103] The "means for capturing the user's facial expression" is a technology for capturing the user's facial expression using a camera or an image capture device and collecting the image data.

[1104] The "means for transmitting image data of license plates and facial expressions to a server" refers to the communication protocols and network technologies for transferring captured image data to a server in real time.

[1105] "Means for extracting license plate information as text data" refers to a technology that uses image analysis technology to extract the numbers and letters on license plates and convert them into text data.

[1106] "Means for analyzing user emotions" refers to technology that uses image analysis and machine learning to identify a user's emotional state from their facial expressions and obtain that information as data.

[1107] The "means for recording the start and end times of parking a vehicle" refers to a technology that automatically records the time when a vehicle enters and exits the parking lot and saves that data.

[1108] The "means for charging based on parking time" is a technology for calculating fees based on recorded parking time information and performing charging processing.

[1109] The "means for adjusting the method of billing notification based on extracted emotional information" is a technique for dynamically changing the content and display method of the notification message in consideration of the emotional state of the user.

[1110] "Means for comparing with a resident database" refers to a technology that compares the collected license plate information with a pre-registered resident information database to confirm whether the vehicle belongs to the resident.

[1111] "Means for providing parking records, video data, and emotion data" refers to technology that stores and searches past parking history, related video data, and emotion recognition results, and provides them in response to inquiries.

[1112] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, it aims to improve the user experience. The main components of this system and their specific operation are described in detail below.

[1113] System configuration

[1114] This system is constructed using the following main hardware and software:

[1115] Hardware

[1116] 1. Camera: Use a camera with high resolution and night vision, such as Sony's high-sensitivity camera.

[1117] 2. Server: Uses high-performance data processing equipment to receive, analyze, and record data.

[1118] software

[1119] 1. License plate recognition software: Detects vehicle license plates and extracts their information as text data. For example, "OpenALPR" is used.

[1120] 2. Emotion engine: Analyzes the user's facial expressions and recognizes their emotions. For example, the "Microsoft Azure Emotion Recognition API" is used.

[1121] 3. Database: A database is used to store parking logs, emotion data, and related video data.

[1122] System Operation

[1123] The device (camera) automatically detects vehicles entering temporary parking spaces and takes images of their license plates. At the same time, it also captures the facial expressions of the users inside the vehicles. This image data is sent to the server in real time.

[1124] The server first uses "OpenALPR" to extract the license plate number as text data from the received image data. At the same time, it uses an emotion engine to analyze facial expression data and recognize the user's emotions. For specific emotion recognition, it uses "Microsoft Azure Emotion Recognition API."

[1125] The server compares the extracted license plate information with the resident database, and if there is a match, it grants permission to park and records the parking start time. When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and facial expression and sends it to the server. The server records the end time, calculates the parking time, and retains the emotion data.

[1126] Charges are processed based on parking time; specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, a charge of 50 yen is applied for every 5 minutes. The method of charge notification is adjusted based on emotional data. For example, if the emotional state is recognized as stressed, a thoughtful message such as "Thank you for your hard work. Your parking time has been increased to 30 minutes. 400 yen has been added to the management fee" is displayed.

[1127] Specific examples

[1128] Example 1: Resident A

[1129] 1. The device (camera) detects the vehicle and captures the license plate number "ABC-1234" and resident A's smiling face.

[1130] 2. The server extracts the license plate number and matches it with "Resident A."

[1131] 3. The server records the parking start time.

[1132] 4. Resident A leaves 10 minutes later, and another photo is taken and the data is sent.

[1133] 5. The server records the end time and calculates the parking time.

[1134] 6. It will be treated as free parking and a friendly message will appear: "Welcome back, your parking was free for 10 minutes!"

[1135] Example 2: Resident B

[1136] 1. The device (camera) detects the vehicle and captures the license plate number "DEF-5678" and Resident B's stressed expression.

[1137] 2. The server extracts the license plate number and matches it with "Resident B."

[1138] 3. The server records the parking start time.

[1139] 4. Resident B departs 30 minutes later, and another image is taken and the data is transmitted.

[1140] 5. The server records the end time and calculates the parking time.

[1141] 6. The charge of 400 yen will be calculated and the message will be displayed: "Thank you for your time. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[1142] Prompt Sentence Examples

[1143] "I would like to create specifications for a temporary parking space management system for my apartment complex. I would like the system to include the following processes: license plate recognition, recording and charging for parking time, and analyzing user sentiment to reflect this in charging notifications. Please explain in detail the specific process flow and the technologies and tools to be used."

[1144] In this way, this system solves the problem of temporary parking in apartment complexes, achieving both convenience and good manners, as well as improving the user experience.

[1145] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1146] Step 1:

[1147] The device (camera) detects vehicles entering a temporary parking space and photographs the license plate and the user's facial expression. Specifically, a high-sensitivity Sony camera is used to capture the vehicle and the user's face. The input is the vehicle entering and the user's face, and the output is image data of the license plate and facial expression. This allows the initial data to be obtained.

[1148] Step 2:

[1149] The license plate and facial expression image data captured by the device (camera) is sent to the server in real time. This data transmission uses the HTTPS protocol. The input is the captured image data, and the output is the data transfer to the server. This ensures that the data is sent to the server securely.

[1150] Step 3:

[1151] The server extracts the license plate number as text data from the image data of the license plate received. Specifically, it uses license plate recognition software such as "OpenALPR." The input is the image data of the license plate, and the output is the extracted text data. This allows the specific license plate number to be obtained.

[1152] Step 4:

[1153] The server simultaneously analyzes the facial expression data using an emotion engine to recognize the user's emotions. This uses the Microsoft Azure Emotion Recognition API. The input is image data of facial expressions, and the output is the user's emotional data, which is the result of the analysis. This allows the user's emotional state to be identified.

[1154] Step 5:

[1155] The server compares the extracted license plate information with a resident database and allows parking if there is a match. This process uses a database query. The input is the text data of the license plate, and the output is the match result (allow or deny). This allows or denies parking.

[1156] Step 6:

[1157] The server records the parking start time. The time is automatically obtained from the system time. The input is the authorized license plate information, and the output is the parking start time data. This records the parking start time.

[1158] Step 7:

[1159] When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and the user's facial expression and sends them to the server. The input is the vehicle leaving the parking space and the user's face, and the output is the image data captured again. This allows the data at the end to be obtained.

[1160] Step 8:

[1161] The server records the license plate number and the end time. The input is the license plate information photographed again and the system time, and the output is the end time data. This records the end time of parking.

[1162] Step 9:

[1163] The server calculates the parking time based on the parking start time and end time. The input is the parking start time and end time, and the output is the parking time data. At the same time, it stores the emotion data provided by the emotion engine.

[1164] Step 10:

[1165] The server processes the charges based on the parking time. Specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, 50 yen is charged for every 5 minutes. The input is the parking time data, and the output is the amount charged. At this time, the notification message is adjusted based on the emotion data. For example, it displays, "Thank you for your hard work. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[1166] Step 11:

[1167] The server stores all parking logs, emotion recognition data, and related video data in a database. The input is various logs and data, and the output is stored in the database, which allows for future inquiries.

[1168] (Application example 2)

[1169] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1170] Factories and logistics facilities require efficient management of temporary vehicle parking spaces while charging appropriate parking fees. It is also necessary to provide a comfortable parking experience by taking drivers' emotions into consideration. However, conventional systems have difficulty not only recognizing license plates and recording parking time, but also evaluating emotions, limiting the improvement of user experience. This creates a need for efficient parking management while reducing driver stress and dissatisfaction.

[1171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1172] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on the parking time, a means including an emotion engine for recognizing the user's emotions, and an artificial intelligence system that interfaces these means, thereby enabling efficient management of temporary vehicle parking spaces and adjustment of parking fees taking into account the driver's emotions.

[1173] A "means for recognizing license plates" is a device or system that uses a camera or image recognition technology to read a vehicle's license plate and extract that information as digital data.

[1174] The "means for recording the start and end times of vehicle parking" is a system that detects the moment a vehicle enters and exits a parking space and automatically records those times.

[1175] The "means for charging based on parking time" is a system for calculating parking fees based on the recorded parking time and charging the appropriate amount.

[1176] "Means including an emotion engine that recognizes the user's emotions" refers to a program or device that uses a camera or sensor to analyze the user's facial expressions and movements and determine their emotions.

[1177] "A system using artificial intelligence to interface these means" is a system equipped with artificial intelligence that integrates each of the above means and controls and manages them to operate as a whole.

[1178] "Means for comparing with a user database to determine whether the vehicle belongs to the user" refers to a system that compares the collected license plate information with an existing database to determine whether the vehicle belongs to a legitimate user.

[1179] The "means for providing stored parking records and video data in response to inquiries" refers to a system that searches for past parking records and video data as needed and responds to users.

[1180] "A means of charging in 5-minute increments for parking exceeding 21 minutes" refers to a system that has the function of charging for the excess parking time in fixed time increments if the parking time exceeds the designated free time.

[1181] The "means for adjusting the method of charge notification based on the user's emotions" is a system that selects an appropriate message and notification method based on the recognized emotional state of the user and notifies the user of the parking fee.

[1182] This invention is a parking space management system for designated areas within a factory, and combines license plate recognition, parking time measurement, charging processing, and an emotion engine that recognizes user emotions. This system uses various hardware and software to operate in an integrated manner.

[1183] The overall system configuration is as follows:

[1184] Hardware used:

[1185] Cameras (used to capture vehicle license plates and driver faces)

[1186] Factory robots (used to manage parking spaces)

[1187] Software used:

[1188] OpenCV (library for processing images from the camera)

[1189] EmotionEngine (software that analyzes user emotions)

[1190] PlateRecognition (license plate recognition software)

[1191] datetime (standard library for measuring parking time)

[1192] System operation overview:

[1193] The server captures the vehicle's license plate and the driver's facial expression through a camera, and recognizes the license plate number and the driver's emotions from the captured data. It converts the license plate information into text data and records the parking start and end times. Based on this, it calculates the parking time and processes the appropriate fee. It also adjusts the fee notification method based on the driver's emotions.

[1194] Examples:

[1195] Example 1: Truck A parks in a designated area within the factory. The camera recognizes the license plate number of truck "TRK-1234" and the smiling face of the driver. The parking time is 18 minutes, and the parking is treated as free parking. A friendly message is displayed.

[1196] Example 2: Truck B parks for 40 minutes. The camera recognizes the license plate number "TRK-5678" and the driver's stressed facial expression. The charging system calculates the 40 minutes at the end of parking and charges 200 yen for the 20 minutes over 21 minutes. A friendly and considerate message is displayed.

[1197] Example prompt sentence:

[1198] "Please recognize the license plate of a truck in a parking space in a designated area within the factory, measure the parking time using a rule-based AI model that analyzes the driver's facial expressions, and calculate the parking fee. Please generate a fee calculation for 45 minutes of parking and a corresponding message if the driver seems stressed."

[1199] In this way, the embodiment of the present invention enables efficient management of parking spaces within a factory and adjustment of parking fees taking into account the driver's emotions.

[1200] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1201] Step 1:

[1202] License plate and facial expression capture

[1203] Input: Video data from the device (camera)

[1204] Specific operation: The device (camera) detects a vehicle entering a parking space, captures an image of the license plate and the driver's facial expression, and transmits this video data to a server in real time.

[1205] Output: License plate image data and facial expression image data

[1206] Step 2:

[1207] License Plate Recognition

[1208] Input: License plate image data

[1209] Specific operation: The server receives the transmitted license plate image data and uses PlateRecognition software to extract the license plate number as text data.

[1210] Output: License plate number text data

[1211] Step 3:

[1212] Emotion recognition

[1213] Input: facial expression image data

[1214] Specific operation: The server receives the transmitted facial expression image data, analyzes the driver's emotions using the EmotionEngine software, and extracts the driver's emotional state as text data.

[1215] Output: Text data of the driver's emotional state

[1216] Step 4:

[1217] Recording parking start time

[1218] Input: License plate number text data and driver emotional state text data

[1219] Specific operation: The server receives the license plate number and emotional state, and records the current time as the parking start time.

[1220] Output: Recorded data of parking start time

[1221] Step 5:

[1222] Recording parking end time

[1223] Input: Video data (recapture) from the device (camera)

[1224] Specific operation: When the vehicle leaves the parking space, the device (camera) captures the license plate number and the driver's facial expression again and sends them to the server. The server recognizes the recaptured license plate number and emotional state and records the current time as the parking end time.

[1225] Output: Recorded data of parking end time

[1226] Step 6:

[1227] Parking time calculation and charging

[1228] Input: Recorded data of parking start time and parking end time

[1229] Specific operation: The server calculates the parking time based on the parking start time and parking end time, and calculates the charge amount according to the parking time. The emotion engine generates a message to adjust the charge notification method based on the driver's emotional state analyzed.

[1230] Output: Charge amount and notification message

[1231] Step 7:

[1232] Billing notifications and logging

[1233] Input: Charge amount and notification message

[1234] Specific operation: The server notifies the driver of the calculated fee and the generated notification message. It also stores all parking logs, emotion recognition data, and related video data in a database and records the logs for future inquiries.

[1235] Output: Notification to the driver and recording data to the database

[1236] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1239] [Fourth embodiment]

[1240] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1241] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1242] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1243] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1244] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1246] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1247] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1248] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1249] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1250] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1251] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1252] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1253] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. The system includes means for recognizing license plates, measuring parking time, and automatically charging.

[1254] The main components of the system are:

[1255] 1. How to recognize license plates

[1256] 2. A means of recording parking start and end times

[1257] 3. Means of charging based on parking time

[1258] 4. AI-based systems that interface these methods

[1259] Overall system operation

[1260] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate. The server analyzes the image data sent and extracts the text data of the license plate. The server then checks the extracted license plate information against a resident database to confirm whether it matches. The server then records the start time of parking, and again the end time when the user's errand is finished. The server then charges based on the parking time, and saves the record.

[1261] License Plate Recognition

[1262] The device (camera) recognizes the vehicle's license plate. The camera captures the license plate image data and sends it to a server in real time. AI on the server analyzes the image data and extracts the license plate number as text data.

[1263] Parking time record

[1264] The server records the parking start time along with the license plate information. When the vehicle parked in the parking space leaves, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time using the start and end time information.

[1265] Billing process

[1266] The server calculates the parking time, charging for parking up to 20 minutes for free, and for parking over 21 minutes for every 5 minutes. This charge is automatically added to the resident's management fee.

[1267] Record keeping and inquiries

[1268] The terminal (server) stores all parking logs and video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs and video data and provide them promptly.

[1269] Specific examples

[1270] Example 1: Resident A parks his car in a temporary parking space. The terminal (camera) takes a photo of the car's license plate number "ABC-1234", and the server extracts the license plate information as text data and records the parking start time. When Resident A returns 10 minutes later, he is photographed again as he leaves the parking space, and the server records the parking end time and processes the parking as free.

[1271] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a photo of the license plate "DEF-5678", and the server calculates the parking time as 30 minutes from the information taken by taking another photo at the end of the parking period. The server calculates the charge as 200 yen for every 5 minutes from the 21st minute onwards (2 times), and adds 400 yen to the management fee.

[1272] In this way, by following this embodiment, the problem of temporary parking in apartment buildings can be solved, and both convenience and good manners can be achieved.

[1273] The processing flow will be explained below.

[1274] Step 1:

[1275] The terminal (camera) detects vehicles entering a temporary parking space and takes an image of the license plate.

[1276] Step 2:

[1277] The image data of the license plate captured by the terminal (camera) is sent to the server in real time.

[1278] Step 3:

[1279] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number (e.g., "ABC-1234") as text data.

[1280] Step 4:

[1281] The server temporarily stores the extracted license plate information together with time information in a database.

[1282] Step 5:

[1283] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[1284] Step 6:

[1285] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[1286] Step 7:

[1287] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures an image of the license plate.

[1288] Step 8:

[1289] The image data of the license plate captured by the terminal (camera) is sent back to the server.

[1290] Step 9:

[1291] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[1292] Step 10:

[1293] The server records the parking end time together with the license plate information in a database.

[1294] Step 11:

[1295] The server checks the parking start time and parking end time and calculates the total parking time.

[1296] Step 12:

[1297] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[1298] Step 13:

[1299] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[1300] Step 14:

[1301] The server stores all parking logs and related video data in a database along with the calculation results.

[1302] Step 15:

[1303] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[1304] Step 16:

[1305] When a user (resident) makes an inquiry, the server searches the stored parking logs and video data and provides them promptly.

[1306] Example 1

[1307] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1308] In apartment complexes, there are problems with unauthorized parking and long-term parking in temporary parking spaces reserved for residents. This problem causes inconvenience to other residents and hinders the efficient use of temporary parking spaces. Furthermore, without a system in place to properly charge based on parking time, there is a high possibility of problems arising due to a lack of fairness and transparency. For this reason, a system is needed to efficiently manage parking spaces, prevent unauthorized parking, and properly charge.

[1309] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1310] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on parking time, a means for analyzing transmitted image data and extracting license plate text data, and a means for comparing the extracted license plate information with a database. This allows for real-time recognition of license plates, accurate recording of parking start and end times, and appropriate charging. It also prevents unauthorized parking and realizes efficient management of temporary parking spaces reserved for residents.

[1311] "Means for recognizing license plates" refers to a device or technology that uses a camera or other device to photograph a vehicle's license plate and identify the license plate information from the image.

[1312] The "means for recording the vehicle parking start and end times" refers to a device or technology for recording the time stamps when a vehicle enters and leaves a parking space.

[1313] The "means for charging based on parking time" refers to a device or technology for calculating parking time based on the recorded parking start time and end time, and for processing charges based on the result.

[1314] "Means for analyzing transmitted image data and extracting license plate text data" refers to a device or technology for processing image data transmitted from a camera or the like and extracting the license plate text information contained in the image in text format.

[1315] "Means for matching extracted license plate information with a database" refers to a device or technique for comparing extracted license plate text data with existing database information to search for matching information.

[1316] The "resident database" is a database for managing information about residents of an apartment complex, and specifically includes information such as license plate information for each resident.

[1317] An "artificial intelligence system" is a system that uses artificial intelligence technology to analyze and process data, specifically automating functions such as license plate recognition and data matching.

[1318] "Means for providing stored parking logs and video data in response to inquiries" refers to a device or technology for quickly searching and providing past parking records and recorded video data in response to inquiries from residents.

[1319] The present invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, preventing unauthorized parking, and automatically charging appropriate fees. This system includes means for recognizing license plates, recording parking time, and charging fees, and is described in detail below.

[1320] Hardware and software used

[1321] The main hardware and software used in this system are as follows:

[1322] Hardware

[1323] IP camera (e.g. Hikvision DS-2CD2085FWD-I)

[1324] software

[1325] OCR (Optical Character Recognition) software (e.g., Google Cloud Vision API)

[1326] AI models (e.g. custom models using TensorFlow)

[1327] Database management software (e.g., MySQL)

[1328] Program processing

[1329] The device (camera) detects vehicles entering temporary parking spaces and captures images of their license plates. The captured image data is sent to a server in real time. The server analyzes the received image data and uses OCR technology to extract the text data from the license plates. This is done using AI models such as Google Cloud Vision API and TensorFlow.

[1330] The server then compares the extracted license plate information with a database to determine whether the vehicle belongs to the resident. Database management software such as MySQL is used for the comparison. Once the verification is complete, the server records the parking start time as a timestamp.

[1331] When the user leaves the parking space after completing their errands, the device (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time by calculating the difference between the parking start time and end time.

[1332] The server processes the charges based on the calculated parking time. Parking is free for up to 20 minutes, but if it exceeds 21 minutes, a charge is made for every 5 minutes. This charge is automatically added to the resident's management fee.

[1333] Specific examples of billing processing

[1334] Example 1: Resident A parks his car in a temporary parking space. The device (camera) takes a photo of the car's license plate number "ABC-1234" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. When Resident A returns 10 minutes later and moves the car, another photo is taken, and the server records the end time of parking and processes the parking as free parking.

[1335] Example 2: Resident B parks for 30 minutes. The terminal (camera) takes a picture of license plate "DEF-5678" and sends the image data to the server. The server extracts the license plate information as text data and records the start time of parking. The photo is taken again 30 minutes later, and the server records the end time of parking. The server calculates the parking time as 30 minutes, charges for 10 minutes from the 21st minute onwards (200 yen x 2 times), and adds 400 yen to the management fee.

[1336] Prompt Sentence Examples

[1337] "Please explain in natural language in detail each processing step of the program's processing flow for a temporary parking management system for an apartment complex: vehicle detection and license plate recognition, converting license plate information to text and matching it with a database, recording the parking start time, recording the parking end time and calculating the parking time, charging, saving the parking log, and responding to inquiries."

[1338] In this way, by implementing the system of the present invention, the management of temporary parking spaces in apartment complexes can be made more efficient, unauthorized parking can be prevented, and appropriate charging can be achieved.

[1339] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1340] Step 1:

[1341] The terminal (camera) detects a vehicle and takes an image of the license plate. The input is a video of a vehicle entering a parking space. When the camera detects a vehicle, it captures an image of the license plate and outputs it as image data. The specific operation that takes place at this point is that the IP camera analyzes the real-time video and captures an image of the license plate when a vehicle enters the space as a trigger.

[1342] Step 2:

[1343] The input is the license plate image data captured by the camera. The terminal (camera) sends this image data to the server. The server receives the image data and proceeds to the next analysis step. The output is the image data of the license plate sent to the server. The specific operation in this step is for the terminal to convert the license plate image captured into a data packet and send it to the server via the network.

[1344] Step 3:

[1345] The input is the license plate image data received by the server. The server analyzes the received image data and extracts the text data of the license plate using OCR technology. Specifically, it uses Google Cloud Vision API and TensorFlow to extract the character information from the image. The output obtained from this process is the text data of the license plate number. The specific operation in this step is that the server inputs the image data into optical character recognition software and obtains the character information in text format.

[1346] Step 4:

[1347] The input is the extracted license plate text data. The server checks this license plate information against a resident database. This is done using a database management system such as MySQL. The output of the database check is the matching resident information, or a result if no matching information is found. The specific operation in this step is that the server queries the database for the license plate text data and searches for matching information.

[1348] Step 5:

[1349] The input is resident information obtained based on the matching. The server recognizes that the resident's vehicle has entered the parking space and records the parking start time as a timestamp. The output is the parking start time data linked to the license plate information. The specific operation in this step is for the server to obtain the current time and record it as the start time in the database.

[1350] Step 6:

[1351] When the user finishes his business and moves his vehicle out of the parking space, the terminal (camera) captures an image of the license plate again. The input is the video of the vehicle leaving the parking space. When the camera detects the vehicle leaving the parking space, it captures the image of the license plate again and outputs it as image data. Specifically, the IP camera captures the image of the license plate again, triggering the vehicle's departure.

[1352] Step 7:

[1353] The input is the re-photographed license plate image data. The terminal (camera) sends this image data to the server. The server analyzes the received image data and re-extracts the license plate text data. The output is the text data of the license plate number at the end of parking. The specific operation is to send the photographed license plate image to the server.

[1354] Step 8:

[1355] The input is the text data of the license plate at the end of parking. The server records the end time and calculates the difference from the start time to calculate the parking time. The output is the parking time data. In concrete terms, the server calculates the difference between the parking start time and the end time and records the result as the parking time.

[1356] Step 9:

[1357] The input is the calculated parking time data. The server processes the charge based on the parking time. Parking up to 20 minutes is free, and if it exceeds 21 minutes, the charge is calculated in 5-minute increments. The output is the final charge information, which is added to the management fee. The specific operation is that the server calculates the charge based on the parking time and adds that amount to the resident's account.

[1358] Step 10:

[1359] The inputs are parking start and end times, license plate information, and billing information. The server stores all parking logs and video data in a database. The output is the stored parking logs and video data. The specific operation is to store this information in the database as structured data so that it can be used to respond to future queries.

[1360] (Application example 1)

[1361] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1362] Conventional parking lot management systems made it difficult to accurately grasp parking lot usage, and were unable to effectively manage charges to increase customer convenience. This resulted in unauthorized parking and illegal parking, increasing store management costs. It was also difficult to respond quickly and accurately to inquiries about parking lot usage. To solve these problems, an effective parking lot management system was needed.

[1363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1364] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of parking of vehicles, a means for charging based on parking time, a means for checking the parking status in real time using a smartphone, and a means for making payments using a smartphone. This automates the management of parking lot usage, making it possible to improve the convenience of customers and the efficiency of parking lot management for stores.

[1365] A "license plate" is an identification plate attached to a vehicle that indicates the registration number.

[1366] "Means of recognition" refers to technology that uses a device such as a camera to detect a vehicle's license plate and read the information.

[1367] "Means for recording parking start and end times" refers to a system that automatically records the times a vehicle enters and leaves a parking lot.

[1368] "Charging means" refers to a system that calculates and charges fees based on parking time.

[1369] "Artificial intelligence" refers to computer programs and algorithms that automate complex processes such as data analysis and image recognition.

[1370] "Means for checking parking status in real time using a smartphone" refers to technology that allows users to grasp the current parking status in real time using a smartphone application.

[1371] "Smartphone payment method" refers to technology that allows users to pay parking fees online using a smartphone application.

[1372] A "resident database" refers to a database that stores information about residents and regular customers of apartment complexes and stores.

[1373] The present invention provides a system that recognizes vehicle license plates, records parking time, and charges at physical stores. This system has the functionality to check parking status in real time and make payments online using a smartphone.

[1374] Hardware Configuration

[1375] This system uses the following hardware:

[1376] Camera: An IP camera installed in the parking lot captures the vehicle's license plate.

[1377] Server: A server (e.g., AWS EC2) for analyzing image data, recording parking time, and processing billing.

[1378] Smartphone: A device that allows users to check parking availability in real time and make payments.

[1379] Software Configuration

[1380] This system uses the following software:

[1381] OpenCV: Image processing library. Used for license plate recognition.

[1382] pytesseract: A library for optical character recognition (OCR) to extract text data from license plates.

[1383] Flask: A web application framework that manages server-side processing.

[1384] React Native: A mobile application development framework used to develop smartphone apps.

[1385] System processing flow

[1386] 1. License Plate Recognition

[1387] The device (camera) takes a photo of the vehicle's license plate and sends the image data to the server. The AI ​​on the server uses OpenCV and pytesseract to analyze the image data and extract the text data from the license plate.

[1388] 2. Parking time record

[1389] The server records the parking start time along with the license plate information. When the vehicle leaves the parking lot, the terminal (camera) takes another image of the license plate and sends it to the server. The server records the parking end time and calculates the parking time.

[1390] 3. Billing Process

[1391] The server processes the charges based on the calculated parking time. Parking up to 20 minutes is free, and for parking over 21 minutes, the fee is calculated in 5-minute increments. The user is then presented with a bill via a smartphone app and can pay online.

[1392] 4. Check parking status

[1393] Users can check the parking situation in real time using their smartphone, which allows them to understand the parking space usage status and remaining time.

[1394] 5. Response to inquiries

[1395] The server stores all parking logs and image data and has the functionality to respond quickly to inquiries from users, which helps prevent problems related to parking records.

[1396] Specific examples

[1397] Example 1

[1398] Customer A parks his car in the store's parking lot. The terminal (camera) takes a photo of the car's license plate, and the server extracts the license plate information as text data and records the parking start time. When Customer A leaves the parking lot 30 minutes later, the terminal (camera) takes another photo of the license plate, and the server records the parking end time. Charges are processed for the 10 minutes beyond 21 minutes, and a fee of 400 yen is displayed on the smartphone app.

[1399] Example prompts to input to the generative AI model

[1400] How can I develop an application that uses a license plate recognition system in a parking lot to automatically measure the parking time of customers and process the charges? Also, please provide the actual Python code.

[1401] In this way, by using the system of the present invention, parking lot management at physical stores can be made significantly more efficient, improving convenience for customers.

[1402] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1403] Step 1:

[1404] Photographing license plates

[1405] How it works: The device's camera detects vehicles entering a parking lot and takes an image of their license plate.

[1406] Input: Vehicle license plate number

[1407] Output: License plate image data

[1408] How it works: IP cameras installed in parking lots detect vehicles using motion detection and automatically capture images of license plates, which are then sent to a server.

[1409] Step 2:

[1410] License plate text analysis

[1411] Operation: The server analyzes the image data of the license plate received and extracts it as text data.

[1412] Input: License plate image data

[1413] Output: License plate text data

[1414] How it works: Image processing is performed on the server using OpenCV and pytesseract. First, the image is converted to grayscale, contour detection is performed, and then the license plate number is converted to text data using optical character recognition (OCR).

[1415] Step 3:

[1416] Recording parking start time

[1417] How it works: The server records the parking start time along with the license plate information in a database.

[1418] Input: License plate text data, parking start time

[1419] Output: Parking start information recorded in the database

[1420] Specific operation: Uses Python's datetime module to obtain the current time and saves it in a database along with the license plate text data.

[1421] Step 4:

[1422] Recording parking end time

[1423] Operation: When the vehicle leaves the parking lot, the device (camera) takes another image of the license plate and sends it to the server. The server analyzes the received image and records the end time.

[1424] Input: License plate image data, parking end time

[1425] Output: Parking end information recorded in the database

[1426] Specific operation: Recognize the license plate from the image using the same method as in step 2, obtain the end time and record it in the database.

[1427] Step 5:

[1428] Parking fee calculation

[1429] Operation: The server calculates the parking time based on the parking start and end times and processes the charges.

[1430] Input: Parking start time, parking end time

[1431] Output: Parking fee calculation result

[1432] Specific operation: Parking time is calculated, and parking for up to 20 minutes is free, but if it exceeds 21 minutes, 200 yen is charged for every 5 minutes. The calculated fee is sent to the user's smartphone app.

[1433] Step 6:

[1434] Real-time confirmation and payment

[1435] How it works: Users use a smartphone app to check parking availability in real time and pay online.

[1436] Input: Parking status data, parking fee data

[1437] Output: Payment completion notification

[1438] Specific operation: The user opens the smartphone app and checks the parking time and calculated fee. After the user makes the payment, the app sends a payment completion notification to the server, which records it.

[1439] Step 7:

[1440] Inquiry response

[1441] Operation: The server stores all parking logs and image data and responds to user inquiries.

[1442] Input: Parking log, image data

[1443] Output: Inquiry response results

[1444] Specific operation: Parking logs and image data are backed up regularly and can be quickly searched and provided as needed, allowing for accurate and prompt responses to user inquiries.

[1445] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1446] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience, and is also combined with an emotion engine that recognizes the user's emotions. In addition to the basic functions of recognizing license plates, measuring parking time, and processing fees, this system can also analyze the user's emotions and reflect that information in the way parking fees are presented.

[1447] The main components of the system are:

[1448] 1. How to recognize license plates

[1449] 2. A means of recording parking start and end times

[1450] 3. Means of charging based on parking time

[1451] 4. AI-based systems that interface these methods

[1452] 5. Emotion engine that recognizes user emotions

[1453] Overall system operation

[1454] The terminal (camera) detects a vehicle entering a temporary parking space and takes an image of the license plate. The terminal (camera) also takes a photo of the user's facial expression, and the emotion engine analyzes the image data to recognize the user's emotions. The server receives the license plate image data and emotion analysis data and extracts the license plate number as text data. The server then compares the extracted license plate information with a resident database to confirm whether it matches, and then records the parking start time and the parking end time again when the user's errand is finished. The server then charges based on the parking time and saves the record.

[1455] License Plate Recognition and Emotion Recognition

[1456] The device (camera) recognizes the vehicle's license plate and captures the user's facial expression. The license plate image data and facial expression data captured by the camera are sent to a server in real time. The AI ​​on the server analyzes the license plate image data and extracts the license plate number as text data. At the same time, the emotion engine analyzes the received facial expression data and determines the user's emotion.

[1457] Parking time record

[1458] The server records the parking start time along with the license plate information. When the vehicle leaves the parking space, the device (camera) captures another image of the license plate and the user's facial expression and sends them to the server. The server records the license plate number and end time, and calculates the parking time and evaluates the emotion based on the parking time and emotion data.

[1459] Billing and Emotional Reflection

[1460] The server calculates the parking time, charging for parking up to 20 minutes free of charge and for parking over 21 minutes in five-minute increments. The server also adjusts the method of notifying the user of the charge (for example, by displaying a friendly message) based on the emotion data provided by the emotion engine. The charge is automatically added to the resident's management fee.

[1461] Record keeping and inquiries

[1462] The terminal (server) stores all parking logs, emotion recognition data, and related video data in a database. In the unlikely event that a resident makes an inquiry, the server can search the stored logs, video data, and emotion recognition data and provide the information promptly.

[1463] Specific examples

[1464] Example 1: Resident A parks his / her car in a temporary parking space. The device (camera) captures the car's license plate number "ABC-1234" and Resident A's facial expression. The server extracts the license plate information as text data, and at the same time, the emotion engine recognizes the smiling expression. The parking time is 10 minutes, so it is treated as free parking, and a friendly breakdown message is displayed.

[1465] Example 2: Resident B parks for 30 minutes. The device (camera) captures the license plate number "DEF-5678" and Resident B's facial expression, and the emotion engine recognizes the stressed expression. The server calculates the parking time as 30 minutes from the information captured again at the end of parking, calculates the charge as 10 minutes from the 21st minute onwards, displays a friendly and considerate message, and adds 400 yen to the management fee.

[1466] In this way, this embodiment solves the problem of temporary parking in apartment buildings, and not only achieves both convenience and good manners, but also improves the user experience.

[1467] The processing flow will be explained below.

[1468] Step 1:

[1469] The device (camera) detects the vehicle entering and simultaneously captures the license plate and the user's facial expression.

[1470] Step 2:

[1471] The license plate image data and facial expression data captured by the terminal (camera) are sent to the server in real time.

[1472] Step 3:

[1473] The license plate recognition AI on the server analyzes the received image data and extracts the license plate number as text data.

[1474] Step 4:

[1475] The server temporarily stores the extracted license plate information together with time information in a database.

[1476] Step 5:

[1477] The emotion engine on the server analyzes the facial expression data, recognizes the user's emotions, and records the information in a database.

[1478] Step 6:

[1479] The server compares the stored license plate information with a resident database to confirm whether it belongs to the relevant resident.

[1480] Step 7:

[1481] The server records the parking start time for the license plate that it has confirmed belongs to a resident.

[1482] Step 8:

[1483] When the user (resident) returns after completing their errands and the vehicle leaves the parking space, the terminal (camera) again captures the license plate and the user's facial expression.

[1484] Step 9:

[1485] The image data of the license plate and facial expression data captured by the terminal (camera) are sent back to the server.

[1486] Step 10:

[1487] The license plate recognition AI on the server re-analyzes the received image data and extracts the license plate number as text data.

[1488] Step 11:

[1489] The emotion engine on the server analyzes the facial expression data again, recognizes the user's emotions, and records that information in a database.

[1490] Step 12:

[1491] The server obtains the parking start time and parking end time from the database and calculates the total parking time.

[1492] Step 13:

[1493] The server determines whether the parking time is within 20 minutes or over 20 minutes. If it is within 20 minutes, it is free, but if it is over 20 minutes, it will be charged in 5-minute increments from the 21st minute onwards.

[1494] Step 14:

[1495] The server calculates the fee based on the overtime, set in 5-minute increments, and automatically adds it to the resident's management fee.

[1496] Step 15:

[1497] The server adjusts the method of billing notification (e.g., message content and display format) based on parking time and emotion recognition data.

[1498] Step 16:

[1499] The server stores all parking logs, emotion recognition data and related video data in a database along with the calculation results.

[1500] Step 17:

[1501] When the user (resident) makes a payment, the parking fee is automatically included in the management fee bill.

[1502] Step 18:

[1503] When a user (resident) makes an inquiry, the server searches the stored parking logs, emotion recognition data, and video data and provides them promptly.

[1504] Example 2

[1505] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] In managing temporary parking spaces reserved for residents in apartment complexes, efficient parking management, improved user convenience, and proper parking etiquette are required. Furthermore, a method that takes into account the user's feelings is also necessary, but existing systems have difficulty achieving these simultaneously.

[1507] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1508] In this invention, the server includes means for recognizing the vehicle's license plate, means for capturing the user's facial expression, means for transmitting image data of the license plate and the facial expression to the server, means for extracting the license plate information as text data, means for analyzing the user's emotions, means for recording the vehicle's parking start time and end time, means for charging based on the parking time, and means for adjusting the method of notifying the charge based on the extracted emotion information, thereby enabling efficient parking management and flexible response that takes the user's emotions into consideration.

[1509] "Means for recognizing vehicle license plates" refers to technology that uses cameras and image analysis software to detect vehicle license plates and extract that information as digital data.

[1510] The "means for capturing the user's facial expression" is a technology for capturing the user's facial expression using a camera or an image capture device and collecting the image data.

[1511] The "means for transmitting image data of license plates and facial expressions to a server" refers to the communication protocols and network technologies for transferring captured image data to a server in real time.

[1512] "Means for extracting license plate information as text data" refers to a technology that uses image analysis technology to extract the numbers and letters on license plates and convert them into text data.

[1513] "Means for analyzing user emotions" refers to technology that uses image analysis and machine learning to identify a user's emotional state from their facial expressions and obtain that information as data.

[1514] The "means for recording the start and end times of parking a vehicle" refers to a technology that automatically records the time when a vehicle enters and exits the parking lot and saves that data.

[1515] The "means for charging based on parking time" is a technology for calculating fees based on recorded parking time information and performing charging processing.

[1516] The "means for adjusting the method of billing notification based on extracted emotional information" is a technique for dynamically changing the content and display method of the notification message in consideration of the emotional state of the user.

[1517] "Means for comparing with a resident database" refers to a technology that compares the collected license plate information with a pre-registered resident information database to confirm whether the vehicle belongs to the resident.

[1518] "Means for providing parking records, video data, and emotion data" refers to technology that stores and searches past parking history, related video data, and emotion recognition results, and provides them in response to inquiries.

[1519] This invention is a system for efficiently managing temporary parking spaces reserved for residents in apartment complexes, achieving both good manners and convenience. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, it aims to improve the user experience. The main components of this system and their specific operation are described in detail below.

[1520] System configuration

[1521] This system is constructed using the following main hardware and software:

[1522] Hardware

[1523] 1. Camera: Use a camera with high resolution and night vision, such as Sony's high-sensitivity camera.

[1524] 2. Server: Uses high-performance data processing equipment to receive, analyze, and record data.

[1525] software

[1526] 1. License plate recognition software: Detects vehicle license plates and extracts their information as text data. For example, "OpenALPR" is used.

[1527] 2. Emotion engine: Analyzes the user's facial expressions and recognizes their emotions. For example, the "Microsoft Azure Emotion Recognition API" is used.

[1528] 3. Database: A database is used to store parking logs, emotion data, and related video data.

[1529] System Operation

[1530] The device (camera) automatically detects vehicles entering temporary parking spaces and takes images of their license plates. At the same time, it also captures the facial expressions of the users inside the vehicles. This image data is sent to the server in real time.

[1531] The server first uses "OpenALPR" to extract the license plate number as text data from the received image data. At the same time, it uses an emotion engine to analyze facial expression data and recognize the user's emotions. For specific emotion recognition, it uses "Microsoft Azure Emotion Recognition API."

[1532] The server compares the extracted license plate information with the resident database, and if there is a match, it grants permission to park and records the parking start time. When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and facial expression and sends it to the server. The server records the end time, calculates the parking time, and retains the emotion data.

[1533] Charges are processed based on parking time; specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, a charge of 50 yen is applied for every 5 minutes. The method of charge notification is adjusted based on emotional data. For example, if the emotional state is recognized as stressed, a thoughtful message such as "Thank you for your hard work. Your parking time has been increased to 30 minutes. 400 yen has been added to the management fee" is displayed.

[1534] Specific examples

[1535] Example 1: Resident A

[1536] 1. The device (camera) detects the vehicle and captures the license plate number "ABC-1234" and resident A's smiling face.

[1537] 2. The server extracts the license plate number and matches it with "Resident A."

[1538] 3. The server records the parking start time.

[1539] 4. Resident A leaves 10 minutes later, and another photo is taken and the data is sent.

[1540] 5. The server records the end time and calculates the parking time.

[1541] 6. It will be treated as free parking and a friendly message will appear: "Welcome back, your parking was free for 10 minutes!"

[1542] Example 2: Resident B

[1543] 1. The device (camera) detects the vehicle and captures the license plate number "DEF-5678" and Resident B's stressed expression.

[1544] 2. The server extracts the license plate number and matches it with "Resident B."

[1545] 3. The server records the parking start time.

[1546] 4. Resident B departs 30 minutes later, and another image is taken and the data is transmitted.

[1547] 5. The server records the end time and calculates the parking time.

[1548] 6. The charge of 400 yen will be calculated and the message will be displayed: "Thank you for your time. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[1549] Prompt Sentence Examples

[1550] "I would like to create specifications for a temporary parking space management system for my apartment complex. I would like the system to include the following processes: license plate recognition, recording and charging for parking time, and analyzing user sentiment to reflect this in charging notifications. Please explain in detail the specific process flow and the technologies and tools to be used."

[1551] In this way, this system solves the problem of temporary parking in apartment complexes, achieving both convenience and good manners, as well as improving the user experience.

[1552] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1553] Step 1:

[1554] The device (camera) detects vehicles entering a temporary parking space and photographs the license plate and the user's facial expression. Specifically, a high-sensitivity Sony camera is used to capture the vehicle and the user's face. The input is the vehicle entering and the user's face, and the output is image data of the license plate and facial expression. This allows the initial data to be obtained.

[1555] Step 2:

[1556] The license plate and facial expression image data captured by the device (camera) is sent to the server in real time. This data transmission uses the HTTPS protocol. The input is the captured image data, and the output is the data transfer to the server. This ensures that the data is sent to the server securely.

[1557] Step 3:

[1558] The server extracts the license plate number as text data from the image data of the license plate received. Specifically, it uses license plate recognition software such as "OpenALPR." The input is the image data of the license plate, and the output is the extracted text data. This allows the specific license plate number to be obtained.

[1559] Step 4:

[1560] The server simultaneously analyzes the facial expression data using an emotion engine to recognize the user's emotions. This uses the Microsoft Azure Emotion Recognition API. The input is image data of facial expressions, and the output is the user's emotional data, which is the result of the analysis. This allows the user's emotional state to be identified.

[1561] Step 5:

[1562] The server compares the extracted license plate information with a resident database and allows parking if there is a match. This process uses a database query. The input is the text data of the license plate, and the output is the match result (allow or deny). This allows or denies parking.

[1563] Step 6:

[1564] The server records the parking start time. The time is automatically obtained from the system time. The input is the authorized license plate information, and the output is the parking start time data. This records the parking start time.

[1565] Step 7:

[1566] When the vehicle leaves the parking space, the device (camera) again captures an image of the license plate and the user's facial expression and sends them to the server. The input is the vehicle leaving the parking space and the user's face, and the output is the image data captured again. This allows the data at the end to be obtained.

[1567] Step 8:

[1568] The server records the license plate number and the end time. The input is the license plate information photographed again and the system time, and the output is the end time data. This records the end time of parking.

[1569] Step 9:

[1570] The server calculates the parking time based on the parking start time and end time. The input is the parking start time and end time, and the output is the parking time data. At the same time, it stores the emotion data provided by the emotion engine.

[1571] Step 10:

[1572] The server processes the charges based on the parking time. Specifically, parking for up to 20 minutes is free, and for parking over 21 minutes, 50 yen is charged for every 5 minutes. The input is the parking time data, and the output is the amount charged. At this time, the notification message is adjusted based on the emotion data. For example, it displays, "Thank you for your hard work. Your parking time is 30 minutes. 400 yen has been added to the management fee."

[1573] Step 11:

[1574] The server stores all parking logs, emotion recognition data, and related video data in a database. The input is various logs and data, and the output is stored in the database, which allows for future inquiries.

[1575] (Application example 2)

[1576] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1577] Factories and logistics facilities require efficient management of temporary vehicle parking spaces while charging appropriate parking fees. It is also necessary to provide a comfortable parking experience by taking drivers' emotions into consideration. However, conventional systems have difficulty not only recognizing license plates and recording parking time, but also evaluating emotions, limiting the improvement of user experience. This creates a need for efficient parking management while reducing driver stress and dissatisfaction.

[1578] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1579] In this invention, the server includes a means for recognizing license plates, a means for recording the start and end times of vehicle parking, a means for charging based on the parking time, a means including an emotion engine for recognizing the user's emotions, and an artificial intelligence system that interfaces these means, thereby enabling efficient management of temporary vehicle parking spaces and adjustment of parking fees taking into account the driver's emotions.

[1580] A "means for recognizing license plates" is a device or system that uses a camera or image recognition technology to read a vehicle's license plate and extract that information as digital data.

[1581] The "means for recording the start and end times of vehicle parking" is a system that detects the moment a vehicle enters and exits a parking space and automatically records those times.

[1582] The "means for charging based on parking time" is a system for calculating parking fees based on the recorded parking time and charging the appropriate amount.

[1583] "Means including an emotion engine that recognizes the user's emotions" refers to a program or device that uses a camera or sensor to analyze the user's facial expressions and movements and determine their emotions.

[1584] "A system using artificial intelligence to interface these means" is a system equipped with artificial intelligence that integrates each of the above means and controls and manages them to operate as a whole.

[1585] "Means for comparing with a user database to determine whether the vehicle belongs to the user" refers to a system that compares the collected license plate information with an existing database to determine whether the vehicle belongs to a legitimate user.

[1586] The "means for providing stored parking records and video data in response to inquiries" refers to a system that searches for past parking records and video data as needed and responds to users.

[1587] "A means of charging in 5-minute increments for parking exceeding 21 minutes" refers to a system that has the function of charging for the excess parking time in fixed time increments if the parking time exceeds the designated free time.

[1588] The "means for adjusting the method of charge notification based on the user's emotions" is a system that selects an appropriate message and notification method based on the recognized emotional state of the user and notifies the user of the parking fee.

[1589] This invention is a parking space management system for designated areas within a factory, and combines license plate recognition, parking time measurement, charging processing, and an emotion engine that recognizes user emotions. This system uses various hardware and software to operate in an integrated manner.

[1590] The overall system configuration is as follows:

[1591] Hardware used:

[1592] Cameras (used to capture vehicle license plates and driver faces)

[1593] Factory robots (used to manage parking spaces)

[1594] Software used:

[1595] OpenCV (library for processing images from the camera)

[1596] EmotionEngine (software that analyzes user emotions)

[1597] PlateRecognition (license plate recognition software)

[1598] datetime (standard library for measuring parking time)

[1599] System operation overview:

[1600] The server captures the vehicle's license plate and the driver's facial expression through a camera, and recognizes the license plate number and the driver's emotions from the captured data. It converts the license plate information into text data and records the parking start and end times. Based on this, it calculates the parking time and processes the appropriate fee. It also adjusts the fee notification method based on the driver's emotions.

[1601] Examples:

[1602] Example 1: Truck A parks in a designated area within the factory. The camera recognizes the license plate number of truck "TRK-1234" and the smiling face of the driver. The parking time is 18 minutes, and the parking is treated as free parking. A friendly message is displayed.

[1603] Example 2: Truck B parks for 40 minutes. The camera recognizes the license plate number "TRK-5678" and the driver's stressed facial expression. The charging system calculates the 40 minutes at the end of parking and charges 200 yen for the 20 minutes over 21 minutes. A friendly and considerate message is displayed.

[1604] Example prompt sentence:

[1605] "Please recognize the license plate of a truck in a parking space in a designated area within the factory, measure the parking time using a rule-based AI model that analyzes the driver's facial expressions, and calculate the parking fee. Please generate a fee calculation for 45 minutes of parking and a corresponding message if the driver seems stressed."

[1606] In this way, the embodiment of the present invention enables efficient management of parking spaces within a factory and adjustment of parking fees taking into account the driver's emotions.

[1607] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1608] Step 1:

[1609] License plate and facial expression capture

[1610] Input: Video data from the device (camera)

[1611] Specific operation: The device (camera) detects a vehicle entering a parking space, captures an image of the license plate and the driver's facial expression, and transmits this video data to a server in real time.

[1612] Output: License plate image data and facial expression image data

[1613] Step 2:

[1614] License Plate Recognition

[1615] Input: License plate image data

[1616] Specific operation: The server receives the transmitted license plate image data and uses PlateRecognition software to extract the license plate number as text data.

[1617] Output: License plate number text data

[1618] Step 3:

[1619] Emotion recognition

[1620] Input: facial expression image data

[1621] Specific operation: The server receives the transmitted facial expression image data, analyzes the driver's emotions using the EmotionEngine software, and extracts the driver's emotional state as text data.

[1622] Output: Text data of the driver's emotional state

[1623] Step 4:

[1624] Recording parking start time

[1625] Input: License plate number text data and driver emotional state text data

[1626] Specific operation: The server receives the license plate number and emotional state, and records the current time as the parking start time.

[1627] Output: Recorded data of parking start time

[1628] Step 5:

[1629] Recording parking end time

[1630] Input: Video data (recapture) from the device (camera)

[1631] Specific operation: When the vehicle leaves the parking space, the device (camera) captures the license plate number and the driver's facial expression again and sends them to the server. The server recognizes the recaptured license plate number and emotional state and records the current time as the parking end time.

[1632] Output: Recorded data of parking end time

[1633] Step 6:

[1634] Parking time calculation and charging

[1635] Input: Recorded data of parking start time and parking end time

[1636] Specific operation: The server calculates the parking time based on the parking start time and parking end time, and calculates the charge amount according to the parking time. The emotion engine generates a message to adjust the charge notification method based on the driver's emotional state analyzed.

[1637] Output: Charge amount and notification message

[1638] Step 7:

[1639] Billing notifications and logging

[1640] Input: Charge amount and notification message

[1641] Specific operation: The server notifies the driver of the calculated fee and the generated notification message. It also stores all parking logs, emotion recognition data, and related video data in a database and records the logs for future inquiries.

[1642] Output: Notification to the driver and recording data to the database

[1643] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1644] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1645] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1646] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1647] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1648] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1649] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1650] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1651] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1652] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1653] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1654] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1655] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1656] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1657] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1658] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1659] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1660] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1661] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1662] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1663] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1664] The following is further disclosed regarding the above embodiment.

[1665] (Claim 1)

[1666] a means for recognizing license plates;

[1667] means for recording the vehicle's parking start and end times;

[1668] means for charging based on parking time;

[1669] A system that includes an artificial intelligence-based system that interfaces with these means.

[1670] (Claim 2)

[1671] means for matching the license plate information with a resident database to determine whether the vehicle belongs to a resident;

[1672] including a means for providing stored parking records and video data in response to an inquiry;

[1673] 10. The system of claim 1.

[1674] (Claim 3)

[1675] This includes measures to make parking free for up to 20 minutes and charging in five-minute increments for parking over 21 minutes.

[1676] 10. The system of claim 1.

[1677] "Example 1"

[1678] (Claim 1)

[1679] a means for recognizing license plates;

[1680] means for recording the vehicle's parking start and end times;

[1681] means for charging based on parking time;

[1682] means for analyzing the transmitted image data and extracting text data of the license plate;

[1683] means for matching the extracted license plate information with a database;

[1684] A system that includes an artificial intelligence-based system that interfaces with these means.

[1685] (Claim 2)

[1686] means for matching the license plate information with a resident database to determine whether the vehicle belongs to a resident;

[1687] including a means to provide stored parking log and video data upon inquiry;

[1688] 10. The system of claim 1.

[1689] (Claim 3)

[1690] This includes measures to make parking free for up to 20 minutes and charging in five-minute increments for parking over 21 minutes.

[1691] 10. The system of claim 1.

[1692] "Application Example 1"

[1693] (Claim 1)

[1694] a means for recognizing license plates;

[1695] means for recording the vehicle's parking start and end times;

[1696] means for charging based on parking time;

[1697] A system using artificial intelligence that interfaces these means,

[1698] A means to check parking status in real time using a smartphone,

[1699] A system including a means for making payments using a smartphone.

[1700] (Claim 2)

[1701] means for matching the license plate information with a resident database to determine whether the vehicle belongs to a resident;

[1702] including a means for providing stored parking records and video data in response to an inquiry;

[1703] 10. The system of claim 1.

[1704] (Claim 3)

[1705] This includes measures to make parking free for up to 20 minutes and charging in five-minute increments for parking over 21 minutes.

[1706] 10. The system of claim 1.

[1707] "Example 2: Combining Emotion Engines"

[1708] (Claim 1)

[1709] means for recognizing the license plate of a vehicle;

[1710] means for capturing a user's facial expression;

[1711] means for transmitting image data of the license plate and facial expression to a server;

[1712] means for extracting license plate information as text data;

[1713] means for analyzing user emotions;

[1714] means for recording the vehicle's parking start and end times;

[1715] means for charging based on parking time;

[1716] a means for adjusting a method of billing notification based on the extracted emotion information;

[1717] A system that includes an artificial intelligence-based system that interfaces with these means.

[1718] (Claim 2)

[1719] means for matching the license plate information with a resident database to determine whether the vehicle belongs to a resident;

[1720] a means for providing stored parking records, video data, and emotion data in response to an inquiry;

[1721] 10. The system of claim 1.

[1722] (Claim 3)

[1723] This includes measures to make parking free for up to 20 minutes and charging in five-minute increments for parking over 21 minutes.

[1724] 10. The system of claim 1.

[1725] "Application example 2 when combining emotion engines"

[1726] (Claim 1)

[1727] a means for recognizing license plates;

[1728] means for recording the vehicle's parking start and end times;

[1729] means for charging based on parking time;

[1730] means including an emotion engine for recognizing an emotion of a user;

[1731] A system that includes an artificial intelligence-based system that interfaces with these means.

[1732] (Claim 2)

[1733] means for comparing the license plate information with a user database to determine whether the vehicle belongs to the user;

[1734] including means for providing stored parking records and video data in response to an inquiry;

[1735] 10. The system of claim 1.

[1736] (Claim 3)

[1737] Parking is free for up to 20 minutes, and if it exceeds 21 minutes, a fee is charged for every 5 minutes.

[1738] means for adjusting the method of billing notification based on user sentiment;

[1739] 10. The system of claim 1. [Explanation of symbols]

[1740] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for recognizing license plates; means for recording the vehicle's parking start and end times; means for charging based on parking time; A system that includes an artificial intelligence-based system that interfaces with these means.

2. means for matching the license plate information with a resident database to determine whether the vehicle belongs to a resident; including a means for providing stored parking records and video data in response to an inquiry; The system of claim 1 .

3. This includes measures to make parking free for up to 20 minutes and charging in five-minute increments for parking over 21 minutes. The system of claim 1 .

Images