System

Abstract

To provide a movable storage device capable of efficiently and safely storing articles even in a limited space.SOLUTION: A terminal means for receiving user authentication information and transmitting the user authentication information to a server, a means for the server to collate the received user authentication information with a database and generate an authentication token, a means for the server to transmit the generated authentication token to the terminal, a means for the server to calculate an optimum movement route of the storage rack and transmit a movement instruction to the storage apparatus, a means for transmitting real-time data when the storage apparatus moves to the server, and a means for displaying the real-time data received from the server on the terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional fixed storage devices are not suitable for use in limited spaces, and it is difficult to efficiently store and retrieve items, making it difficult to make effective use of the space. Furthermore, the operation and safety management of the storage device are complicated, resulting in low user convenience. The present invention aims to solve these problems and provide a movable storage device that can store items efficiently and safely even in limited spaces. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a terminal means for receiving user authentication information and transmitting it to a server, a means for the server to compare the received user authentication information with a database and generate an authentication token, a means for transmitting the authentication token generated by the server to the terminal, a means for transmitting user-specified storage shelf destination information from the terminal to the server, a means for the server to calculate an optimal storage shelf movement path and transmit movement instructions to the storage device, a means for transmitting real-time data as the storage device moves to the server, a means for displaying the real-time data received from the server on the terminal, and a means for the server to periodically check the status of safety devices and generate an error message and notify the terminal if an abnormality is detected. This system achieves a highly safe and convenient storage device, with storage shelves that move horizontally and vertically, made of lightweight materials, and equipped with multiple rollers and guide rails. The server also includes an algorithm for calculating the movement path and notifying the user if an abnormality is detected.

[0006] "User authentication information" refers to identification information such as an ID and password used by a user when accessing a system.

[0007] "Terminal means" refers to the device used by the user (smartphone, tablet, PC, etc.), and is the means through which communication with the server is carried out.

[0008] A "server" is a central control unit that processes user authentication information, manages data, sends instructions, and so on.

[0009] "Database" means a collection of data that stores user authentication information and other related information and that is accessed and verified by the Server.

[0010] An "authentication token" is temporary identification information that is generated by the server when a user is successfully authenticated and is used to maintain a session and authorize access.

[0011] A "storage shelf" is a movable shelf for storing items, designed to efficiently utilize limited space.

[0012] "Destination information" is information about the location and direction to which the user wants to move the storage shelf.

[0013] The "movement route" is the route along which the storage shelf is moved safely and efficiently to the designated location.

[0014] "Real-time data" refers to information such as the position, speed, and surrounding obstacles that is acquired in real time as the storage shelf moves.

[0015] "Safety devices" are sensors and other safety mechanisms that detect obstacles while the storage shelf is moving and prevent accidents or malfunctions.

[0016] An "error message" is a warning message that the system generates and notifies the user when a safety device detects an abnormality.

[0017] "Rollers" are wheel-like devices that allow the storage shelves to move smoothly, and move along guide rails.

[0018] A "guide rail" is a rail-like part that allows a storage shelf to move safely along a specific track.

[0019] An "algorithm" is a set of procedures or formulas used by the server to calculate a travel path. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] System configuration

[0042] The system of the present invention comprises a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and set up the storage device.

[0043] Program processing overview and specific examples

[0044] User Authentication

[0045] 1. The user enters their ID and password on the terminal.

[0046] The user enters their ID and password on the app's login screen and presses the login button.

[0047] 2. The device encrypts the user's input and sends it to the server.

[0048] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0049] 3. The server checks the received ID and password against the database.

[0050] The server searches the database for the corresponding user information and, if there is a match, generates an authentication token.

[0051] 4. The server sends the authentication token to the device.

[0052] The server sends the generated authentication token to the terminal, which stores it.

[0053] Determining the location of storage shelves

[0054] 1. The user specifies the destination of the storage shelf on the terminal.

[0055] Users use the in-app interface to select where they want to move the storage shelf.

[0056] 2. The device sends destination information to the server.

[0057] The terminal sends the specified destination information to the server.

[0058] 3. The server calculates the optimal route based on the received information.

[0059] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0060] 4. The server sends a move instruction to the storage device.

[0061] The server sends a movement instruction to the storage device according to the calculated movement route.

[0062] Shelf movement and control

[0063] 1. The server receives real-time data from the storage device.

[0064] When the storage device starts moving, real-time data from sensors and encoders is sent to the server.

[0065] 2. The server sends real-time data to the device.

[0066] The server organizes the received real-time data and sends it to the terminal.

[0067] 3. The terminal displays the current status to the user.

[0068] The terminal displays real-time data received from the server, showing the user the current location and status of the storage shelves.

[0069] 4. The user changes the operation as needed.

[0070] Users can use the app to change their destination or instructions while on the move.

[0071] Checking the operation of safety devices

[0072] 1. The server checks the status of the safety device at regular intervals.

[0073] The server periodically checks the status of sensors and safety devices in the storage device.

[0074] 2. The server generates an error message when an abnormality is detected.

[0075] If the server detects an abnormality, it generates an error message such as "Obstacle detected."

[0076] 3. The server sends an error message to the terminal.

[0077] The server immediately sends any error messages it generates to the terminal.

[0078] 4. The terminal notifies the user of the error message.

[0079] When the device receives an error message, it will notify the user using the app's notification function.

[0080] 5. The user receives a notification and takes corrective action.

[0081] The user receives a notification and can stop the operation or check safety if necessary.

[0082] This system allows users to efficiently and safely store items in limited spaces. Its horizontally and vertically movable shelves and lightweight materials make it highly convenient. Its high load-bearing capacity and customizable modular structure allow it to meet a variety of needs. Integrated safety devices and anomaly detection mechanisms ensure peace of mind.

[0083] The processing flow will be explained below.

[0084] Program processing steps

[0085] User Authentication

[0086] Step 1:

[0087] The user enters their ID and password on the login screen of the device.

[0088] The user launches the app and enters their ID and password on the login screen.

[0089] Step 2:

[0090] The device encrypts the ID and password and sends them to the server.

[0091] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0092] Step 3:

[0093] The server checks the ID and password against the database.

[0094] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[0095] Step 4:

[0096] The server generates an authentication token and sends it to the device.

[0097] The server sends the generated authentication token to the terminal, which stores it.

[0098] Determining the location of storage shelves

[0099] Step 1:

[0100] The user specifies the destination of the storage shelf on the terminal.

[0101] Users select the location where they want to move the storage shelf using the app's interface.

[0102] Step 2:

[0103] The terminal transmits the designated destination information to the server.

[0104] The terminal transmits the destination information specified by the user to the server.

[0105] Step 3:

[0106] The server calculates the optimal travel route based on the information received.

[0107] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0108] Step 4:

[0109] The server sends a movement instruction to the storage device.

[0110] The server sends a movement instruction to the storage device based on the calculated optimum route.

[0111] Shelf movement and control

[0112] Step 1:

[0113] A server receives real-time data from the storage device.

[0114] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[0115] Step 2:

[0116] The server transmits the received real-time data to the terminal.

[0117] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[0118] Step 3:

[0119] The terminal displays the real-time data to the user.

[0120] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[0121] Step 4:

[0122] The user changes the operation as needed.

[0123] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[0124] Checking the operation of safety devices

[0125] Step 1:

[0126] The server checks the status of the safety device at regular intervals.

[0127] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[0128] Step 2:

[0129] The server generates an error message when an abnormality is detected.

[0130] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[0131] Step 3:

[0132] The server sends an error message to the terminal.

[0133] The server immediately sends any error messages it generates to the terminal.

[0134] Step 4:

[0135] The terminal notifies the user of the error message.

[0136] The device will display the error message to the user using the app's notification function.

[0137] Step 5:

[0138] The user receives a notification and takes safety measures.

[0139] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[0140] This allows the user to operate the mobile storage device efficiently and safely.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional storage equipment systems have had issues with safety and efficiency. Unauthorized access, breakdowns, and malfunctions are likely to occur during processes such as user authentication, moving storage shelves by position specification, real-time status display, and safety device operation confirmation, making it difficult to provide a system that users can use with confidence. Therefore, there is a demand for a system that allows users to operate storage equipment easily and safely.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes a terminal means for receiving user authentication information and transmitting it to the server, a means for the terminal to encrypt the user's input and transmit it to the server, and a means for the server to compare the received user authentication information with a database and generate an authentication token, thereby preventing unauthorized access and providing a secure authentication process.

[0146] "User authentication information" refers to information such as an ID and password that a user enters to access a system.

[0147] A "server" is a computer system that collates user authentication information, generates authentication tokens, and controls storage devices.

[0148] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive authentication information and destination information.

[0149] A "database" is a collection of data that a server uses to verify user authentication information.

[0150] An "authentication token" is a temporary digital certificate that indicates that a user has been properly authenticated.

[0151] A "storage shelf" is a device that moves to a specified position to store or retrieve items.

[0152] "Destination information" is location information of the destination of the storage shelf designated by the user.

[0153] "Real-time data" is data that is updated immediately to indicate the current location and status of a storage shelf.

[0154] A "safety device" is a device that detects obstacles and stops the storage shelf safely while it is in operation.

[0155] An "error message" is a warning notification that is generated when a safety device detects an abnormality.

[0156] An "algorithm" is a series of calculation procedures that the server uses to calculate the optimal movement path of the storage shelf.

[0157] The "Dijkstra algorithm" is an algorithm based on graph theory that the server uses to calculate the shortest path through the storage shelves.

[0158] The system of the present invention is composed of a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and configure the storage device, thereby enabling safe and efficient storage and retrieval of items.

[0159] First, the user starts the authentication process through the device application. The user enters their ID and password and presses the login button. The device encrypts the entered authentication information using the SSL / TLS protocol and sends it to the server. The server compares the received authentication information with its internal database and, if a match is found, generates an authentication token. This authentication token is sent to the device and stored on the device.

[0160] After authentication, the user can use the application interface to specify the destination of the storage shelf. The specified destination information is sent from the terminal to the server, and the server calculates the optimal movement route. This calculation uses the Dijkstra algorithm to calculate the shortest route that takes into account obstacles between the current storage shelf position and the specified destination. The server then sends the calculated route instructions to the storage device, and the storage device begins moving along the specified route.

[0161] During movement, real-time data from the storage device's sensors and encoders is sent to the server. The server organizes the received data and sends it to the terminal via the WebSocket protocol. The terminal displays the data in real time, showing the user the current location and status of the storage shelf. If necessary, the user can change the movement destination or instructions.

[0162] Furthermore, the operation of safety devices is checked periodically. The server checks data from the storage device's sensors at regular intervals to confirm the status of the safety devices. If an abnormality is detected, the server generates an error message such as "Obstacle detected" and immediately notifies the terminal. After receiving the notification, the user can take the necessary safety measures.

[0163] As a concrete example, if a user wants to move a storage shelf to "Position A," the device sends the information "Destination: Position A" to the server, and the server calculates the optimal route using the Dijkstra algorithm. Also, if an obstacle is detected while the storage shelf is moving, the sensor sends the data "Obstacle detected: Position X, Y" to the server, and an error message is generated.

[0164] Example prompt for a generative AI model:

[0165] "Please explain the SSL / TLS encryption used for login authentication. Please provide a detailed explanation including specific implementation examples."

[0166] The system allows users to efficiently and safely store and retrieve items in limited spaces, and its high load-bearing capacity and customizable modular structure make it suitable for a variety of needs.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Step 1: The user enters their ID and password.

[0169] The user opens the application on the terminal and enters their ID and password. The input is made into the text boxes on the login screen. The ID (e.g., "user123") and password (e.g., "password123") are recorded on the terminal as input data.

[0170] Step 2: The device encrypts the input and sends it to the server.

[0171] The terminal encrypts the ID and password entered by the user using the SSL / TLS protocol. The encrypted data is sent to the server using an HTTP POST request. At this time, the input data, ID and password, are converted into encrypted form.

[0172] Step 3: The server checks the database.

[0173] The server decrypts the received encrypted data and compares it with its internal database. Specifically, it searches the database for the corresponding user information and checks whether it matches the entered ID and password. If the comparison is successful, it sets a flag indicating "authentication successful."

[0174] Step 4: The server generates and sends an authentication token.

[0175] The server generates an authentication token based on the matching result. This authentication token is in JWT (JSON Web Token) format and contains the user's authentication information. The generated token is sent from the server to the terminal as an HTTP response, which notifies the user that authentication was successful.

[0176] Step 5: The user specifies the movement position.

[0177] The user uses the terminal application interface to specify the destination of the storage shelf. The "destination location information" (e.g., "Location A") is recorded as input data on the terminal. This information is treated as a movement command.

[0178] Step 6: The terminal sends the destination information to the server.

[0179] The terminal sends the specified destination information to the server using an HTTP POST request. The "destination information" as input data is sent to the server and serves as a trigger to start processing.

[0180] Step 7: The server calculates the optimal route.

[0181] The server calculates the optimal route based on the current storage shelf location and the specified destination. Specifically, it calculates the shortest route using the Dijkstra algorithm. The "current location" and "destination location" are processed as input data, and the "optimal route" is obtained as output.

[0182] Step 8: The server sends a transfer instruction to the storage device.

[0183] The server formats the calculated optimal route and sends a movement instruction to the storage device. The route information is sent to the storage device using an HTTP POST request, which causes the storage device to start moving.

[0184] Step 9: The server receives the real-time data.

[0185] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The "current location information" is sent to the server in real time as input data. This allows the current location of the storage shelf to be tracked.

[0186] Step 10: The server sends the data to the terminal.

[0187] The server organizes the received real-time data and sends it to the terminal using the WebSocket protocol. The "organized real-time data" is output to the terminal as input data, allowing the terminal to display the current status.

[0188] Step 11: The device will display the current status.

[0189] The terminal displays the received real-time data in a graphical user interface. The "real-time data" input is drawn on the screen, showing the user the current location and status of the storage shelves.

[0190] Step 12: The user changes the operation.

[0191] The user can check the location of the storage shelf during movement using the app and change the destination or instructions as necessary. The "new destination information" is recorded as input data on the device and then sent back to the server.

[0192] Step 13: The server checks the status.

[0193] The server periodically checks the data sent from the storage device's sensors to confirm the status of the safety devices. The "sensor information" is taken in as input data and a judgment is made as to whether the device is normal or abnormal.

[0194] Step 14: The server generates an error message.

[0195] If abnormal data is received from the sensor, the server generates an error message. The "abnormal information" is processed as input data, and an "error message" is created as output data. Specifically, a message such as "Obstacle detected" is generated.

[0196] Step 15: The server sends an error message to the terminal.

[0197] The server immediately sends the generated error message to the terminal. The "error message" is sent to the terminal as input data and notified in real time.

[0198] Step 16: The terminal notifies the user.

[0199] The terminal notifies the user of the error message by displaying a pop-up message, etc. The "error message" is displayed as input data, and the user is informed of the abnormality.

[0200] Step 17: The user takes safety measures.

[0201] After receiving the notification, the user can take the necessary safety measures, such as stopping the movement of the storage shelf in the app. This action allows the problem to be safely resolved.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] In logistics centers, to improve the efficiency of storing and retrieving goods, it is necessary to quickly and accurately control the position of storage shelves and storage equipment. However, conventional systems lack the ability to acquire real-time data and check the operation of safety devices, making it difficult to improve efficiency and safety. In addition, there is a need to ensure security and optimality in user authentication and route calculation.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage device destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage device and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and generate an error message and notify the terminal if an abnormality is detected, means for using a JSON web token for login, and means for the server to calculate a movement route for the storage device, including a route calculation algorithm when transmitting instructions to the robot to the system, and notifying the user of an error message in real time as a safety measure. This enables efficient storage management through real-time data acquisition and highly safe operation.

[0207] "User authentication information" refers to information used by a user when accessing a system, and includes, for example, a user ID and password.

[0208] "Terminal means" refers to a device or apparatus that is operated by a user and that communicates with a server to send and receive data.

[0209] A "database" is a structured information collection system for managing and storing data used by an organization or system.

[0210] An "authentication token" is a code that indicates that a user has been authenticated, and is used for session management and access control.

[0211] "Storage equipment" refers to equipment or devices for storing and retrieving items.

[0212] "Destination information" is data relating to the location to which the storage equipment should be moved.

[0213] The "movement route" is the optimum route for the storage equipment to reach the designated destination.

[0214] "Real-time data" is up-to-date information about ongoing processes and conditions.

[0215] "Safety equipment" refers to devices and sensors used to ensure work safety.

[0216] An "error message" is a notification message that is generated when the system detects an abnormality or error.

[0217] "JSON Web Token" is a compact representation format used for user authentication and secure exchange of information.

[0218] "Storage equipment" means equipment or devices for the safe storage of items.

[0219] A "route calculation algorithm" is a method or calculation means for calculating the optimal route to a specified destination.

[0220] To implement this invention, a system including a server, a terminal, and a storage device is required. A specific embodiment of this system will be described below.

[0221] The server first manages the user authentication process. The user enters their ID and password on the device, which then encrypts them using the SSL / TLS protocol and sends them to the server. The server compares the received ID and password with its database, and if they match, generates a JWT (JSON Web Token) and sends it to the device. This achieves secure authentication.

[0222] Next, the destination information of the storage device specified by the user is sent from the terminal to the server. The server uses a route calculation algorithm to calculate the optimal route based on the current location of the storage device and the specified destination information. The server sends this route information to the storage device and issues a movement instruction.

[0223] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The server organizes this data and sends it to the terminal, which then displays the movement status to the user in real time. The user can change the movement destination or instructions as needed.

[0224] To enhance safety, the server periodically checks the status of safety devices. If an abnormality is detected, the server immediately generates an error message and notifies the terminal. The terminal receives this and notifies the user. The user receives the notification and can take safety measures.

[0225] For example, when a user requests on a terminal that "I want to move the shelf to position A3," the server calculates the optimal movement path and sends a movement command to the robot. If an obstacle is detected during movement, an error message is immediately generated and notified to the user.

[0226] Example prompt sentence:

[0227] "The user specified that they wanted the shelf to move to the A3 position."

[0228] This system enables efficient storage management and highly safe operation through real-time data acquisition.

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] The user enters their ID and password on the device.

[0232] Input: User ID, Password

[0233] Output: Encrypted user ID and password

[0234] The terminal encrypts the ID and password entered by the user on the login screen using the SSL / TLS protocol and sends it to the server in a secure format.

[0235] Step 2:

[0236] The server checks the received user ID and password against the database.

[0237] Input: Encrypted user ID and password

[0238] Output: Authentication token

[0239] The server decodes the received user ID and password and checks them against the information stored in the database. If they match, the server generates a JWT (JSON Web Token).

[0240] Step 3:

[0241] The server sends the generated authentication token to the terminal.

[0242] Input: Authentication Token

[0243] Output: Authentication token sent to the device

[0244] The server sends the generated JWT to the device, which stores it and uses it for subsequent authentication.

[0245] Step 4:

[0246] The destination information of the storage device designated by the user is transmitted from the terminal to the server.

[0247] Input: Destination information (e.g., A3 position)

[0248] Output: Destination information sent to the server

[0249] The user uses the application interface to specify the destination of the storage device, and the terminal transmits this information to the server.

[0250] Step 5:

[0251] The server calculates the optimal movement route for the storage device and sends a movement instruction to the storage device.

[0252] Input: Destination information, current storage device location

[0253] Output: Optimal travel route, travel instructions

[0254] The server uses the current location and the specified destination information to execute a route calculation algorithm to calculate the optimal route, and then sends this movement instruction to the storage device.

[0255] Step 6:

[0256] Real-time data on the movement of stored equipment is sent to the server.

[0257] Input: Real-time data of storage equipment

[0258] Output: Real-time data sent to the server

[0259] As the storage equipment begins to move, real-time data from sensors and encoders is collected and sent to a server.

[0260] Step 7:

[0261] The real-time data received from the server is displayed on the terminal.

[0262] Input: Real-time data

[0263] Output: Real-time data displayed on the terminal

[0264] The server organizes the received real-time data and sends it to the terminal, which displays the data to the user in real time.

[0265] Step 8:

[0266] The server periodically checks the status of the safety equipment and, if there is an abnormality, generates an error message and notifies the terminal.

[0267] Input: Safety equipment status data

[0268] Output: Error message, user notification

[0269] The server periodically checks the status of the safety equipment, and if an abnormality is detected, it generates an error message that is immediately sent to the terminal, which then notifies the user.

[0270] Step 9:

[0271] The user receives a notification and takes safety measures.

[0272] Input: Error message

[0273] Output: User's response action

[0274] When the terminal notifies the user of an error message, the user receives it and can stop or change the operation as needed.

[0275] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0276] System configuration

[0277] The system of the present invention is composed of a user, a terminal, a server, and an emotion engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion engine recognizes the user's emotions and adjusts the system's response based on these.

[0278] Program processing overview and specific examples

[0279] User Authentication

[0280] 1. The user enters their ID and password on the device's login screen.

[0281] The user enters their ID and password on the app's login screen and presses the login button.

[0282] 2. The device encrypts the ID and password and sends them to the server.

[0283] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0284] 3. The server checks the ID and password against the database.

[0285] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[0286] 4. The server generates an authentication token and sends it to the device.

[0287] The server sends the generated authentication token to the terminal, which stores it.

[0288] Determining the location of storage shelves

[0289] 1. The user specifies the destination of the storage shelf on the terminal.

[0290] Users select the location where they want to move the storage shelf using the app's interface.

[0291] 2. The terminal sends the specified destination information to the server.

[0292] The terminal transmits the destination information specified by the user to the server.

[0293] 3. The server calculates the optimal route based on the received information.

[0294] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0295] 4. The server sends a move instruction to the storage device.

[0296] The server sends a movement instruction to the storage device based on the calculated optimum route.

[0297] Shelf movement and control

[0298] 1. The server receives real-time data from the storage device.

[0299] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[0300] 2. The server sends the received real-time data to the terminal.

[0301] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[0302] 3. The terminal displays the real-time data to the user.

[0303] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[0304] 4. The user changes the operation as needed.

[0305] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[0306] Checking the operation of safety devices

[0307] 1. The server checks the status of the safety device at regular intervals.

[0308] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[0309] 2. The server generates an error message when an abnormality is detected.

[0310] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[0311] 3. The server sends an error message to the terminal.

[0312] The server immediately sends any error messages it generates to the terminal.

[0313] 4. The terminal notifies the user of the error message.

[0314] The device will display the error message to the user using the app's notification function.

[0315] 5. The user receives a notification and takes corrective action.

[0316] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[0317] Implementing the Emotion Engine

[0318] 1. The device recognizes the user's emotions.

[0319] The device's camera, microphone, and other sensors are used to recognize emotions from the user's facial expressions, tone of voice, etc.

[0320] 2. The device sends the emotion data to the emotion engine.

[0321] The device sends the collected emotion data to the emotion engine.

[0322] 3. The emotion engine analyzes the emotion data.

[0323] The emotion engine analyzes the received data to determine the user's current emotional state.

[0324] 4. The server determines the necessary system response based on the emotion engine's analysis results.

[0325] Based on the data from the emotion engine, the server determines an appropriate response, such as simplifying the operating procedure if the user is feeling stressed.

[0326] 5. The server sends the response to the device.

[0327] The server sends an appropriate response to the terminal, which displays the response to the user.

[0328] 6. The user takes action based on the device's response.

[0329] The user can view the response from the terminal and take appropriate action based on it.

[0330] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[0331] The processing flow will be explained below.

[0332] Program processing steps

[0333] User Authentication

[0334] Step 1:

[0335] The user enters their ID and password on the login screen of the device.

[0336] The user enters their ID and password on the app's login screen and presses the login button.

[0337] Step 2:

[0338] The device encrypts the ID and password and sends them to the server.

[0339] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0340] Step 3:

[0341] The server checks the ID and password against the database.

[0342] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[0343] Step 4:

[0344] The server generates an authentication token and sends it to the device.

[0345] The server sends the generated authentication token to the terminal, which stores it.

[0346] Determining the location of storage shelves

[0347] Step 1:

[0348] The user specifies the destination of the storage shelf on the terminal.

[0349] The user selects a specific location within the app's interface where they want to move the storage shelf, such as "to the left of the living room."

[0350] Step 2:

[0351] The terminal transmits the designated destination information to the server.

[0352] The device sends destination information specified by the user to the server, including the coordinates and direction of the destination.

[0353] Step 3:

[0354] The server calculates the optimal travel route based on the information received.

[0355] The server calculates the optimal route based on the current location of the storage shelf and the specified destination, using the A algorithm or Dijkstra algorithm.

[0356] Step 4:

[0357] The server sends a movement instruction to the storage device.

[0358] The server then sends movement instructions to the storage device based on the calculated optimal route, including start and stop times for the movement, and alternative routes.

[0359] Shelf movement and control

[0360] Step 1:

[0361] A server receives real-time data from the storage device.

[0362] When the storage device starts moving, it sends real-time data from sensors and encoders to the server, including its current position, moving speed, and encoder counts.

[0363] Step 2:

[0364] The server transmits the received real-time data to the terminal.

[0365] The server organizes real-time data received from the storage device and sends it to the user's device, including the device's movement trajectory and current location information.

[0366] Step 3:

[0367] The terminal displays the real-time data to the user.

[0368] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves to the user. For example, the storage shelf icon can be moved in real time on the app.

[0369] Step 4:

[0370] The user changes the operation as needed.

[0371] Users can use the app to change the storage shelf's destination or stop it while it is moving, for example, by issuing commands such as "Stop moving" or "Move to another location."

[0372] Checking the operation of safety devices

[0373] Step 1:

[0374] The server checks the status of the safety device at regular intervals.

[0375] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule, with check intervals set to about 10 seconds.

[0376] Step 2:

[0377] The server generates an error message when an abnormality is detected.

[0378] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected," along with the type and location of the abnormality.

[0379] Step 3:

[0380] The server sends an error message to the terminal.

[0381] The server immediately sends any error messages it generates to the terminal.

[0382] Step 4:

[0383] The terminal notifies the user of the error message.

[0384] The device will display the error message to the user using the app's notification feature, which may include sound and / or vibration.

[0385] Step 5:

[0386] The user receives a notification and takes safety measures.

[0387] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[0388] Implementing the Emotion Engine

[0389] Step 1:

[0390] The device recognizes the user's emotions.

[0391] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice, and sends them to the emotion engine. Evaluation items include smiles, tone of voice, and facial muscle movements.

[0392] Step 2:

[0393] The device transmits the emotion data to the emotion engine.

[0394] The device transmits the collected emotion data to the emotion engine in real time.

[0395] Step 3:

[0396] The emotion engine analyzes the emotion data.

[0397] The emotion engine analyzes the received data with a proprietary algorithm to identify the user's current emotional state, specifically using a machine learning model to classify emotional states such as "stressed" or "relaxed."

[0398] Step 4:

[0399] The server determines the necessary system response based on the analysis results of the emotion engine.

[0400] Based on the results of the emotion engine, the server determines an appropriate response, such as simplifying the operation procedure if the user is feeling stressed.

[0401] Step 5:

[0402] The server sends a response to the terminal.

[0403] The server sends an appropriate response to the terminal, which displays the response to the user.

[0404] Step 6:

[0405] The user takes action according to the terminal's response.

[0406] The user can view the response from the device and take appropriate action based on it, for example by displaying a suggestion such as "Start a relaxing massage."

[0407] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] Current mobile storage systems have basic functions such as user authentication, moving storage shelves, displaying real-time data, and monitoring safety devices, but they lack system responses that take user emotions into consideration, and there is a need for improved user experience. Another issue is the lack of a mechanism for quickly notifying users when an abnormality is detected. This can lead to complex operations and insufficient safety.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server includes means for comparing user authentication information with a database and generating an authentication token, means for calculating an optimal movement route for the storage shelf, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and send the emotion data to an analysis engine, and means for the analysis engine to analyze the emotion data, the server to determine a system response, and display the response on the terminal. This provides an appropriate system response that takes the user's emotion into consideration, making it possible to simplify operation and improve safety.

[0413] "User authentication information" refers to information such as ID and password required for a user to log in to a system.

[0414] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive various data.

[0415] A "server" is a central computer system that connects to a database, performs various calculations and data processing, and sends instructions and information to terminals and other devices.

[0416] An "authentication token" is electronic information that is generated after the server has confirmed the authentication of a user, and is used to maintain the authentication status of the user.

[0417] A "storage shelf" is a movable storage device, which is an item storage device that moves to a specified position based on a user's instructions.

[0418] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[0419] The "optimal movement route" is the most efficient route from the current storage shelf position to the movement destination specified by the user.

[0420] "Real-time data" refers to data that is updated immediately, such as location information and sensor information generated when a storage device moves.

[0421] "Safety devices" are sensors and mechanical protection devices that ensure safety during operation of the storage device.

[0422] An "error message" is a warning or error message that is generated when the system detects an abnormality and is notified to the user.

[0423] "Emotion data" refers to data related to the user's emotions, such as facial expressions and tone of voice, acquired through the device's sensors and camera.

[0424] An "analysis engine" is software or hardware for analyzing emotion data to identify the user's emotional state.

[0425] "System response" refers to the system's actions and display messages that the server determines based on the user's operations and emotional state.

[0426] The system of the present invention is composed of a user, a terminal, a server, and an emotion analysis engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion analysis engine recognizes the user's emotions and adjusts the system's response based on these.

[0427] Hardware and Software Configuration

[0428] Specific examples of system hardware include smartphones and PCs as terminals. These terminals are equipped with cameras, microphones, touchscreens, etc., and provide a means of interface with users. The server uses a high-performance database management system (e.g., MySQL, PostgreSQL), and cloud services can also be used.

[0429] The system's software implementation uses the following technologies:

[0430] Use SSL / TLS encryption (e.g., OpenSSL library) for user authentication.

[0431] RESTful API and WebSocket are used to send and receive data.

[0432] The optimal movement path of the storage shelf is calculated using Dijkstra's algorithm or A algorithm (e.g., NetworkX library).

[0433] Use a JavaScript library (e.g. D3.js) to display real-time data.

[0434] The Python Scikit-learn library is used to check the operation of various safety devices and detect abnormalities.

[0435] OpenCV and Google Cloud Speech API are used to recognize user emotions, and TensorFlow and PyTorch are used to analyze emotion data.

[0436] Example of a system

[0437] Here is a specific example of how the system works. When a user wants to move a storage shelf from the "living room" to the "office," they perform the following operations:

[0438] 1. The user selects the "Office" area on the smartphone app interface and taps the Go button.

[0439] 2. The device sends the selected destination information to the server as an HTTP POST request.

[0440] 3. The server uses the NetworkX library to calculate the optimal route using Dijkstra's algorithm and sends movement instructions to the storage device using the MQTT protocol.

[0441] 4. When the storage device starts moving, it sends real-time data (location information and sensor information) to the server via the MQTT protocol.

[0442] 5. The server organizes the received real-time data and sends it to the user's device using WebSocket.

[0443] 6. The device uses D3.js to visualize the received data on the app screen, showing the user the current location and status of the storage shelf.

[0444] 7. If the user feels tired or stressed while traveling, the device's camera and microphone will collect emotional data and send it to the emotion analysis engine.

[0445] 8. The emotion analysis engine uses TensorFlow to analyze the emotion data and notify the server that the user is feeling stressed.

[0446] 9. Based on the emotion analysis results, the server generates a guide message that simplifies the operation procedure and sends it to the device via WebSocket.

[0447] 10. The user can operate the storage device efficiently by following the guide messages from the terminal and performing the necessary operations.

[0448] Prompt Sentence Examples

[0449] Possible prompts for a generative AI model include:

[0450] "How can I design an application that automatically adjusts its system response based on user emotion?"

[0451] "Please give us a concrete example of an algorithm that uses an emotion analysis engine to detect a user's stress level and simplify the operation procedure."

[0452] This allows the user to safely and efficiently operate the movable storage device while receiving appropriate support according to their emotional state.

[0453] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0454] Step 1:

[0455] The user enters their ID and password on the login screen of the device.

[0456] Input: ID and password entered by the user

[0457] How it works: The user enters their ID and password on the login screen of their smartphone or computer and clicks the login button.

[0458] Output: Device encrypted credentials

[0459] Step 2:

[0460] The device encrypts the entered ID and password and sends them to the server.

[0461] Input: ID and password entered by the user

[0462] How it works: The device encrypts the ID and password using the SSL / TLS protocol (e.g., using the OpenSSL library) and sends it to the server as an HTTP POST request.

[0463] Output: Encrypted credentials sent to the server

[0464] Step 3:

[0465] The server checks the ID and password against the database.

[0466] Input: Encrypted credentials

[0467] What it does: The server accesses a database (e.g. MySQL or PostgreSQL) and runs a SQL query to verify that the entered ID and password match.

[0468] Output: Authentication result (success or failure)

[0469] Step 4:

[0470] The server generates an authentication token and sends it to the device.

[0471] Input: If authentication result is successful

[0472] How it works: The server generates an authentication token using a JWT (JSON Web Token) library and sends it to the device as an HTTP response. The device stores this token in local storage.

[0473] Output: Authentication token

[0474] Step 5:

[0475] The user specifies the destination of the storage shelf on the terminal.

[0476] Input: User-selected destination information

[0477] How it works: The user selects the new location for the storage shelf in the in-app interface and clicks a button to confirm the move.

[0478] Output: Destination information

[0479] Step 6:

[0480] The terminal transmits the designated destination information to the server.

[0481] Input: Destination information

[0482] Operation: The device sends the specified destination information in JSON format to the server. This is done using a RESTful API as an HTTP POST request.

[0483] Output: Destination information is sent to the server

[0484] Step 7:

[0485] The server calculates the optimal travel route based on the information received.

[0486] Input: Current location and specified destination information

[0487] How it works: The server uses Dijkstra's algorithm or A algorithm (using Python's NetworkX library, for example) to calculate the optimal route from the current location to the specified destination.

[0488] Output: Optimal route information

[0489] Step 8:

[0490] The server sends a movement instruction to the storage device.

[0491] Input: Optimal route information

[0492] Operation: The server sends the calculated optimal route information to the storage device. This is done using the MQTT protocol.

[0493] Output: Movement instructions are sent to the storage device

[0494] Step 9:

[0495] Real-time data is sent to the server as the storage device moves.

[0496] Input: Movement instructions

[0497] Operation: The storage device sends position and status information obtained from motors and sensors to a server. MQTT is used as the communication protocol.

[0498] Output: Real-time data

[0499] Step 10:

[0500] The real-time data received by the server is displayed on the terminal.

[0501] Input: Real-time data

[0502] Operation: The server organizes the received data and sends it to the device in real time using WebSocket.

[0503] Output: Organized real-time data

[0504] Step 11:

[0505] The terminal displays the real-time data to the user.

[0506] Input: Organized real-time data

[0507] Operation: The device uses the JavaScript D3.js library to display the received real-time data as graphs and maps on the app screen.

[0508] Output: Visualized current shelf location and status

[0509] Step 12:

[0510] The user changes the operation as needed.

[0511] Input: Real-time data and current operating status

[0512] How it works: The user can change the position and speed of the storage shelf through the app interface. The changed instructions are immediately sent to the server.

[0513] Output: Modified operating instructions

[0514] Step 13:

[0515] The server periodically checks the status of the safety device.

[0516] Input: Safety device sensor information

[0517] How it works: The server periodically schedules tasks to obtain information from safety sensors, for example, using Crontab or a system daemon.

[0518] Output: Safety device status data

[0519] Step 14:

[0520] The server generates an error message when an abnormality is detected.

[0521] Input: Safety device status data

[0522] How it works: The server uses Python's Scikit-learn library to perform anomaly detection and generates an error message if an anomaly is detected.

[0523] Output: Error message

[0524] Step 15:

[0525] The server sends an error message to the terminal.

[0526] Input: Error message

[0527] How it works: The server uses a RESTful API to send an error message to the device.

[0528] Output: Error messages are sent to the terminal

[0529] Step 16:

[0530] The terminal notifies the user of the error message.

[0531] Input: Error message

[0532] What happens: The device sends a push notification to the user using Firebase Cloud Messaging (FCM).

[0533] Output: Error message to the user

[0534] Step 17:

[0535] The user receives a notification and takes safety measures.

[0536] Input: Error message

[0537] Action: The user checks the error message and takes the necessary action (for example, restarting the storage device or performing manual operations).

[0538] Output: Execution of safety instructions

[0539] Step 18:

[0540] The device recognizes the user's emotions.

[0541] Input: User's face and voice data

[0542] How it works: The device uses a camera and microphone to collect user emotions in real time, using OpenCV and Google Cloud Speech API.

[0543] Output: Emotion data

[0544] Step 19:

[0545] The device sends the emotion data to the emotion analysis engine.

[0546] Input: Emotion data

[0547] Operation: The device sends the collected emotion data to the analysis engine via a RESTful API.

[0548] Output: Emotion data is sent to the analysis engine

[0549] Step 20:

[0550] The sentiment analysis engine analyzes the sentiment data.

[0551] Input: Emotion data

[0552] How it works: The sentiment analysis engine uses TensorFlow and PyTorch to analyze emotion data and identify the user's emotional state.

[0553] Output: Emotional state data

[0554] Step 21:

[0555] The server determines the necessary system response based on the emotion analysis results.

[0556] Input: Emotional state data

[0557] Action: The server determines the system response (e.g., simplifying the operation procedure or changing the message) based on the results of the sentiment analysis.

[0558] Output: System response

[0559] Step 22:

[0560] The server sends a response to the terminal.

[0561] Input: System Response

[0562] Operation: The server sends the determined response to the device via WebSocket.

[0563] Output: System responses are sent to the terminal

[0564] Step 23:

[0565] The user takes action according to the terminal's response.

[0566] Input: System Response

[0567] Action: The user checks the response from the device and takes appropriate action based on it. For example, they can simplify the operation by following the guide message.

[0568] Output: Taking appropriate action

[0569] (Application example 2)

[0570] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0571] Conventional storage equipment control systems have issues with operability and convenience because they cannot respond to user operations in real time or take into account the user's stress or emotional state. Furthermore, because the system cannot respond flexibly to emotions, it is unable to respond adequately when the user feels stressed.

[0572] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0573] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage shelf destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage shelf and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and transmit the emotion data to an emotion analysis device, and means for the emotion analysis device to analyze the emotion data and adjust the system response based on the analysis result, thereby enabling a flexible system response according to the user's emotional state.

[0574] "User authentication information" refers to information used to authenticate a user to a server, such as a user ID and password.

[0575] The "terminal means" is a device operated by a user and has functions for performing various operations such as transmitting authentication information and specifying the destination of a storage shelf.

[0576] An "authentication token" is a temporary identifier for identifying a user that is generated by a server after the server has successfully authenticated the user.

[0577] A "storage shelf" is a shelf for storing items, and has a structure that allows it to be moved under the control of the system.

[0578] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[0579] "Emotion recognition" refers to a device analyzing a user's facial expressions, tone of voice, etc. to identify the user's current emotional state.

[0580] "Emotion data" is data relating to the user's emotional state obtained through emotion recognition.

[0581] An "emotion analysis device" is a device that analyzes emotion data and identifies the user's emotional state.

[0582] "System response" refers to the reaction or feedback provided by the system in response to the user's actions or emotional state.

[0583] "Real-time data" refers to data relating to the current location and status of a storage device that is acquired when the storage device is moving.

[0584] "Safety devices" are sensors and control devices that detect obstacles and abnormalities while the storage device is in operation and ensure safety.

[0585] An "error message" is a message that is displayed to the user when the system detects an abnormality.

[0586] The present invention is directed to realizing an in-vehicle equipment management application for an autonomous vehicle, and provides flexible system responses according to the user's emotional state. Specific embodiments of the present invention will be described below.

[0587] System configuration

[0588] The system consists of a terminal operated by the user using a smartphone, a server inside the vehicle, a storage device, and an emotion analysis device. The terminal runs an application for remotely controlling the storage shelves inside the vehicle and also has the ability to recognize the user's emotions.

[0589] Program processing overview and specific examples

[0590] 1. User Authentication

[0591] The user enters their ID and password into the smartphone application and sends them to the server. The server compares the encrypted authentication information with a database, and if authentication is successful, it generates a token and sends it to the terminal. Using this token, the user can continue operating the vehicle.

[0592] 2. Storage device control

[0593] The user specifies the destination of the storage shelf through the application interface. The specified location information is sent to the server, which calculates the optimal movement route and sends the instructions to the storage device. The storage device then sends its location information to the server in real time, and the device displays this information to the user, allowing the user to check the current status of the storage shelf.

[0594] 3. Check safety equipment

[0595] The server periodically checks the status of the safety devices of the storage device, and if an abnormality is detected, it generates an error message and sends it to the terminal. The user receives the notification and can correct or stop the operation as necessary.

[0596] 4. Emotion Recognition and Response

[0597] The system recognizes the user's emotions using the device's camera and microphone. Emotional data is sent to an emotion analysis device, and the system's response is adjusted based on the analysis results. For example, if the user is feeling stressed, the system will simplify the operation procedure and change the in-car music and lighting to a more relaxing environment.

[0598] Hardware and software used

[0599] Hardware

[0600] Smartphone (camera, microphone, internet connection)

[0601] In-vehicle server (data processing and control)

[0602] Storage equipment (mechanical equipment for physically moving items)

[0603] Sentiment analyzer (using a sentiment analysis library like EmotionRecognition)

[0604] software

[0605] Authentication library (encryption and authentication token generation)

[0606] Emotion Analysis Library (EmotionRecognition)

[0607] Specific examples

[0608] For example, suppose a user feels very stressed after finishing an important meeting in the car. The system uses the device's camera and microphone to analyze the user's emotions, and if it determines that the user is stressed, it will automatically dim the lights and play relaxing music. At the same time, when the user operates the storage shelves in the car, the system automatically simplifies each operation to ensure smooth operation.

[0609] Prompt Sentence Examples

[0610] "We will build an application that recognizes user emotions. We will also implement a function to adjust the in-car environment based on the user's emotions. For example, if the user is feeling stressed, we will set relaxing music and lighting."

[0611] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0612] Step 1:

[0613] The user launches the application on their smartphone and enters their ID and password.

[0614] Input: User ID and password

[0615] Processing: The terminal encrypts the ID and password using SSL / TLS.

[0616] Output: Encrypted authentication data

[0617] Step 2:

[0618] The terminal transmits the encrypted authentication data to the server.

[0619] Input: Encrypted authentication data

[0620] Processing: The device sends data to the server's authentication API endpoint.

[0621] Output: Authentication request to the server

[0622] Step 3:

[0623] The server compares the received authentication data with a database and generates an authentication token.

[0624] Input: Encrypted authentication data

[0625] Processing: The server searches the database for the corresponding user information, and if a match is found, generates an authentication token.

[0626] Output: Authentication token

[0627] Step 4:

[0628] The server sends the generated authentication token to the terminal.

[0629] Input: Authentication Token

[0630] Processing: The server returns the authentication token to the terminal as an HTTP response.

[0631] Output: Authentication token received on the device

[0632] Step 5:

[0633] The user specifies the destination of the storage shelf in the application.

[0634] Input: Destination information (location coordinates, etc.)

[0635] Process: The user enters the destination in the application interface and presses the send button.

[0636] Output: Destination information data

[0637] Step 6:

[0638] The terminal transmits destination information data to the server.

[0639] Input: Destination information data

[0640] Processing: The terminal sends destination information to the server.

[0641] Output: Send destination information to the server

[0642] Step 7:

[0643] The server calculates the optimal movement route based on the current location of the storage shelf and the specified destination.

[0644] Input: Current location and destination information

[0645] Processing: The server calculates the optimal travel route using a route calculation algorithm.

[0646] Output: Optimal travel route data

[0647] Step 8:

[0648] The server transmits a movement instruction to the storage device based on the optimum movement route.

[0649] Input: Optimal travel route data

[0650] Processing: The server sends a move instruction to the control API of the storage device.

[0651] Output: Movement instructions to storage device

[0652] Step 9:

[0653] The storage device begins to move and transmits its location data to the server in real time.

[0654] Input: Storage device location data

[0655] Processing: The storage device sends the data acquired from the sensors and encoders to the server.

[0656] Output: Sending real-time data to a server

[0657] Step 10:

[0658] The server organizes the received real-time data and sends it to the terminal.

[0659] Input: Real-time location data

[0660] Processing: The server analyzes and organizes the data and sends it to the device as updated information.

[0661] Output: Sending real-time data to the terminal

[0662] Step 11:

[0663] The terminal displays the received real-time data to the user.

[0664] Input: Real-time location data

[0665] Processing: The device displays real-time data in the application UI.

[0666] Output: Display current location to user

[0667] Step 12:

[0668] The terminal recognizes the user's emotions and transmits the emotion data to an emotion analysis device.

[0669] Input: Emotional data such as the user's facial expressions and tone of voice

[0670] Processing: Emotion data is collected using the device's camera and microphone and sent to an emotion analyzer.

[0671] Output: Sending emotional data to the emotion analyzer

[0672] Step 13:

[0673] An emotion analyzer analyzes the emotion data and adjusts the system response based on the analysis results.

[0674] Input: Emotion data

[0675] Processing: The emotion analyzer analyzes the collected data to identify the user's emotional state.

[0676] Output: Instructions for adjusting system response

[0677] Step 14:

[0678] The server adjusts the system response and sends the adjustment result to the terminal.

[0679] Input: Instructions for adjusting system response

[0680] Processing: Based on the results of emotion analysis, the server performs actions such as simplifying operation procedures or changing the in-car environment (music, lighting, etc.).

[0681] Output: Tuned system response

[0682] Step 15:

[0683] The terminal presents the adjusted system response to the user.

[0684] Input: Tuned system response

[0685] Processing: The terminal presents the adjusted system response to the user through the application.

[0686] Output: Display of tuned system response to user

[0687] This allows the user to safely and efficiently operate the storage device while receiving appropriate support according to their emotional state.

[0688] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0689] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0690] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0691] [Second embodiment]

[0692] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0693] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0694] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0695] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0696] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0697] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0698] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0699] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0700] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0701] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0702] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0703] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0704] System configuration

[0705] The system of the present invention comprises a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and set up the storage device.

[0706] Program processing overview and specific examples

[0707] User Authentication

[0708] 1. The user enters their ID and password on the terminal.

[0709] The user enters their ID and password on the app's login screen and presses the login button.

[0710] 2. The device encrypts the user's input and sends it to the server.

[0711] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0712] 3. The server checks the received ID and password against the database.

[0713] The server searches the database for the corresponding user information and, if there is a match, generates an authentication token.

[0714] 4. The server sends the authentication token to the device.

[0715] The server sends the generated authentication token to the terminal, which stores it.

[0716] Determining the location of storage shelves

[0717] 1. The user specifies the destination of the storage shelf on the terminal.

[0718] Users use the in-app interface to select where they want to move the storage shelf.

[0719] 2. The device sends destination information to the server.

[0720] The terminal sends the specified destination information to the server.

[0721] 3. The server calculates the optimal route based on the received information.

[0722] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0723] 4. The server sends a move instruction to the storage device.

[0724] The server sends a movement instruction to the storage device according to the calculated movement route.

[0725] Shelf movement and control

[0726] 1. The server receives real-time data from the storage device.

[0727] When the storage device starts moving, real-time data from sensors and encoders is sent to the server.

[0728] 2. The server sends real-time data to the device.

[0729] The server organizes the received real-time data and sends it to the terminal.

[0730] 3. The terminal displays the current status to the user.

[0731] The terminal displays real-time data received from the server, showing the user the current location and status of the storage shelves.

[0732] 4. The user changes the operation as needed.

[0733] Users can use the app to change their destination or instructions while on the move.

[0734] Checking the operation of safety devices

[0735] 1. The server checks the status of the safety device at regular intervals.

[0736] The server periodically checks the status of sensors and safety devices in the storage device.

[0737] 2. The server generates an error message when an abnormality is detected.

[0738] If the server detects an abnormality, it generates an error message such as "Obstacle detected."

[0739] 3. The server sends an error message to the terminal.

[0740] The server immediately sends any error messages it generates to the terminal.

[0741] 4. The terminal notifies the user of the error message.

[0742] When the device receives an error message, it will notify the user using the app's notification function.

[0743] 5. The user receives a notification and takes corrective action.

[0744] The user receives a notification and can stop the operation or check safety if necessary.

[0745] This system allows users to efficiently and safely store items in limited spaces. Its horizontally and vertically movable shelves and lightweight materials make it highly convenient. Its high load-bearing capacity and customizable modular structure allow it to meet a variety of needs. Integrated safety devices and anomaly detection mechanisms ensure peace of mind.

[0746] The processing flow will be explained below.

[0747] Program processing steps

[0748] User Authentication

[0749] Step 1:

[0750] The user enters their ID and password on the login screen of the device.

[0751] The user launches the app and enters their ID and password on the login screen.

[0752] Step 2:

[0753] The device encrypts the ID and password and sends them to the server.

[0754] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0755] Step 3:

[0756] The server checks the ID and password against the database.

[0757] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[0758] Step 4:

[0759] The server generates an authentication token and sends it to the device.

[0760] The server sends the generated authentication token to the terminal, which stores it.

[0761] Determining the location of storage shelves

[0762] Step 1:

[0763] The user specifies the destination of the storage shelf on the terminal.

[0764] Users select the location where they want to move the storage shelf using the app's interface.

[0765] Step 2:

[0766] The terminal transmits the designated destination information to the server.

[0767] The terminal transmits the destination information specified by the user to the server.

[0768] Step 3:

[0769] The server calculates the optimal travel route based on the information received.

[0770] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0771] Step 4:

[0772] The server sends a movement instruction to the storage device.

[0773] The server sends a movement instruction to the storage device based on the calculated optimum route.

[0774] Shelf movement and control

[0775] Step 1:

[0776] A server receives real-time data from the storage device.

[0777] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[0778] Step 2:

[0779] The server transmits the received real-time data to the terminal.

[0780] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[0781] Step 3:

[0782] The terminal displays the real-time data to the user.

[0783] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[0784] Step 4:

[0785] The user changes the operation as needed.

[0786] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[0787] Checking the operation of safety devices

[0788] Step 1:

[0789] The server checks the status of the safety device at regular intervals.

[0790] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[0791] Step 2:

[0792] The server generates an error message when an abnormality is detected.

[0793] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[0794] Step 3:

[0795] The server sends an error message to the terminal.

[0796] The server immediately sends any error messages it generates to the terminal.

[0797] Step 4:

[0798] The terminal notifies the user of the error message.

[0799] The device will display the error message to the user using the app's notification function.

[0800] Step 5:

[0801] The user receives a notification and takes safety measures.

[0802] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[0803] This allows the user to operate the mobile storage device efficiently and safely.

[0804] Example 1

[0805] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0806] Conventional storage equipment systems have had issues with safety and efficiency. Unauthorized access, breakdowns, and malfunctions are likely to occur during processes such as user authentication, moving storage shelves by position specification, real-time status display, and safety device operation confirmation, making it difficult to provide a system that users can use with confidence. Therefore, there is a demand for a system that allows users to operate storage equipment easily and safely.

[0807] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0808] In this invention, the server includes a terminal means for receiving user authentication information and transmitting it to the server, a means for the terminal to encrypt the user's input and transmit it to the server, and a means for the server to compare the received user authentication information with a database and generate an authentication token, thereby preventing unauthorized access and providing a secure authentication process.

[0809] "User authentication information" refers to information such as an ID and password that a user enters to access a system.

[0810] A "server" is a computer system that collates user authentication information, generates authentication tokens, and controls storage devices.

[0811] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive authentication information and destination information.

[0812] A "database" is a collection of data that a server uses to verify user authentication information.

[0813] An "authentication token" is a temporary digital certificate that indicates that a user has been properly authenticated.

[0814] A "storage shelf" is a device that moves to a specified position to store or retrieve items.

[0815] "Destination information" is location information of the destination of the storage shelf designated by the user.

[0816] "Real-time data" is data that is updated immediately to indicate the current location and status of a storage shelf.

[0817] A "safety device" is a device that detects obstacles and stops the storage shelf safely while it is in operation.

[0818] An "error message" is a warning notification that is generated when a safety device detects an abnormality.

[0819] An "algorithm" is a series of calculation procedures that the server uses to calculate the optimal movement path of the storage shelf.

[0820] The "Dijkstra algorithm" is an algorithm based on graph theory that the server uses to calculate the shortest path through the storage shelves.

[0821] The system of the present invention is composed of a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and configure the storage device, thereby enabling safe and efficient storage and retrieval of items.

[0822] First, the user starts the authentication process through the device application. The user enters their ID and password and presses the login button. The device encrypts the entered authentication information using the SSL / TLS protocol and sends it to the server. The server compares the received authentication information with its internal database and, if a match is found, generates an authentication token. This authentication token is sent to the device and stored on the device.

[0823] After authentication, the user can use the application interface to specify the destination of the storage shelf. The specified destination information is sent from the terminal to the server, and the server calculates the optimal movement route. This calculation uses the Dijkstra algorithm to calculate the shortest route that takes into account obstacles between the current storage shelf position and the specified destination. The server then sends the calculated route instructions to the storage device, and the storage device begins moving along the specified route.

[0824] During movement, real-time data from the storage device's sensors and encoders is sent to the server. The server organizes the received data and sends it to the terminal via the WebSocket protocol. The terminal displays the data in real time, showing the user the current location and status of the storage shelf. If necessary, the user can change the movement destination or instructions.

[0825] Furthermore, the operation of safety devices is checked periodically. The server checks data from the storage device's sensors at regular intervals to confirm the status of the safety devices. If an abnormality is detected, the server generates an error message such as "Obstacle detected" and immediately notifies the terminal. After receiving the notification, the user can take the necessary safety measures.

[0826] As a concrete example, if a user wants to move a storage shelf to "Position A," the device sends the information "Destination: Position A" to the server, and the server calculates the optimal route using the Dijkstra algorithm. Also, if an obstacle is detected while the storage shelf is moving, the sensor sends the data "Obstacle detected: Position X, Y" to the server, and an error message is generated.

[0827] Example prompt for a generative AI model:

[0828] "Please explain the SSL / TLS encryption used for login authentication. Please provide a detailed explanation including specific implementation examples."

[0829] The system allows users to efficiently and safely store and retrieve items in limited spaces, and its high load-bearing capacity and customizable modular structure make it suitable for a variety of needs.

[0830] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0831] Step 1: The user enters their ID and password.

[0832] The user opens the application on the terminal and enters their ID and password. The input is made into the text boxes on the login screen. The ID (e.g., "user123") and password (e.g., "password123") are recorded on the terminal as input data.

[0833] Step 2: The device encrypts the input and sends it to the server.

[0834] The terminal encrypts the ID and password entered by the user using the SSL / TLS protocol. The encrypted data is sent to the server using an HTTP POST request. At this time, the input data, ID and password, are converted into encrypted form.

[0835] Step 3: The server checks the database.

[0836] The server decrypts the received encrypted data and compares it with its internal database. Specifically, it searches the database for the corresponding user information and checks whether it matches the entered ID and password. If the comparison is successful, it sets a flag indicating "authentication successful."

[0837] Step 4: The server generates and sends an authentication token.

[0838] The server generates an authentication token based on the matching result. This authentication token is in JWT (JSON Web Token) format and contains the user's authentication information. The generated token is sent from the server to the terminal as an HTTP response, which notifies the user that authentication was successful.

[0839] Step 5: The user specifies the movement position.

[0840] The user uses the terminal application interface to specify the destination of the storage shelf. The "destination location information" (e.g., "Location A") is recorded as input data on the terminal. This information is treated as a movement command.

[0841] Step 6: The terminal sends the destination information to the server.

[0842] The terminal sends the specified destination information to the server using an HTTP POST request. The "destination information" as input data is sent to the server and serves as a trigger to start processing.

[0843] Step 7: The server calculates the optimal route.

[0844] The server calculates the optimal route based on the current storage shelf location and the specified destination. Specifically, it calculates the shortest route using the Dijkstra algorithm. The "current location" and "destination location" are processed as input data, and the "optimal route" is obtained as output.

[0845] Step 8: The server sends a transfer instruction to the storage device.

[0846] The server formats the calculated optimal route and sends a movement instruction to the storage device. The route information is sent to the storage device using an HTTP POST request, which causes the storage device to start moving.

[0847] Step 9: The server receives the real-time data.

[0848] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The "current location information" is sent to the server in real time as input data. This allows the current location of the storage shelf to be tracked.

[0849] Step 10: The server sends the data to the terminal.

[0850] The server organizes the received real-time data and sends it to the terminal using the WebSocket protocol. The "organized real-time data" is output to the terminal as input data, allowing the terminal to display the current status.

[0851] Step 11: The device will display the current status.

[0852] The terminal displays the received real-time data in a graphical user interface. The "real-time data" input is drawn on the screen, showing the user the current location and status of the storage shelves.

[0853] Step 12: The user changes the operation.

[0854] The user can check the location of the storage shelf during movement using the app and change the destination or instructions as necessary. The "new destination information" is recorded as input data on the device and then sent back to the server.

[0855] Step 13: The server checks the status.

[0856] The server periodically checks the data sent from the storage device's sensors to confirm the status of the safety devices. The "sensor information" is taken in as input data and a judgment is made as to whether the device is normal or abnormal.

[0857] Step 14: The server generates an error message.

[0858] If abnormal data is received from the sensor, the server generates an error message. The "abnormal information" is processed as input data, and an "error message" is created as output data. Specifically, a message such as "Obstacle detected" is generated.

[0859] Step 15: The server sends an error message to the terminal.

[0860] The server immediately sends the generated error message to the terminal. The "error message" is sent to the terminal as input data and notified in real time.

[0861] Step 16: The terminal notifies the user.

[0862] The terminal notifies the user of the error message by displaying a pop-up message, etc. The "error message" is displayed as input data, and the user is informed of the abnormality.

[0863] Step 17: The user takes safety measures.

[0864] After receiving the notification, the user can take the necessary safety measures, such as stopping the movement of the storage shelf in the app. This action allows the problem to be safely resolved.

[0865] (Application example 1)

[0866] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0867] In logistics centers, to improve the efficiency of storing and retrieving goods, it is necessary to quickly and accurately control the position of storage shelves and storage equipment. However, conventional systems lack the ability to acquire real-time data and check the operation of safety devices, making it difficult to improve efficiency and safety. In addition, there is a need to ensure security and optimality in user authentication and route calculation.

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0869] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage device destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage device and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and generate an error message and notify the terminal if an abnormality is detected, means for using a JSON web token for login, and means for the server to calculate a movement route for the storage device, including a route calculation algorithm when transmitting instructions to the robot to the system, and notifying the user of an error message in real time as a safety measure. This enables efficient storage management through real-time data acquisition and highly safe operation.

[0870] "User authentication information" refers to information used by a user when accessing a system, and includes, for example, a user ID and password.

[0871] "Terminal means" refers to a device or apparatus that is operated by a user and that communicates with a server to send and receive data.

[0872] A "database" is a structured information collection system for managing and storing data used by an organization or system.

[0873] An "authentication token" is a code that indicates that a user has been authenticated, and is used for session management and access control.

[0874] "Storage equipment" refers to equipment or devices for storing and retrieving items.

[0875] "Destination information" is data relating to the location to which the storage equipment should be moved.

[0876] The "movement route" is the optimum route for the storage equipment to reach the designated destination.

[0877] "Real-time data" is up-to-date information about ongoing processes and conditions.

[0878] "Safety equipment" refers to devices and sensors used to ensure work safety.

[0879] An "error message" is a notification message that is generated when the system detects an abnormality or error.

[0880] "JSON Web Token" is a compact representation format used for user authentication and secure exchange of information.

[0881] "Storage equipment" means equipment or devices for the safe storage of items.

[0882] A "route calculation algorithm" is a method or calculation means for calculating the optimal route to a specified destination.

[0883] To implement this invention, a system including a server, a terminal, and a storage device is required. A specific embodiment of this system will be described below.

[0884] The server first manages the user authentication process. The user enters their ID and password on the device, which then encrypts them using the SSL / TLS protocol and sends them to the server. The server compares the received ID and password with its database, and if they match, generates a JWT (JSON Web Token) and sends it to the device. This achieves secure authentication.

[0885] Next, the destination information of the storage device specified by the user is sent from the terminal to the server. The server uses a route calculation algorithm to calculate the optimal route based on the current location of the storage device and the specified destination information. The server sends this route information to the storage device and issues a movement instruction.

[0886] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The server organizes this data and sends it to the terminal, which then displays the movement status to the user in real time. The user can change the movement destination or instructions as needed.

[0887] To enhance safety, the server periodically checks the status of safety devices. If an abnormality is detected, the server immediately generates an error message and notifies the terminal. The terminal receives this and notifies the user. The user receives the notification and can take safety measures.

[0888] For example, when a user requests on a terminal that "I want to move the shelf to position A3," the server calculates the optimal movement path and sends a movement command to the robot. If an obstacle is detected during movement, an error message is immediately generated and notified to the user.

[0889] Example prompt sentence:

[0890] "The user specified that they wanted the shelf to move to the A3 position."

[0891] This system enables efficient storage management and highly safe operation through real-time data acquisition.

[0892] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0893] Step 1:

[0894] The user enters their ID and password on the device.

[0895] Input: User ID, Password

[0896] Output: Encrypted user ID and password

[0897] The terminal encrypts the ID and password entered by the user on the login screen using the SSL / TLS protocol and sends it to the server in a secure format.

[0898] Step 2:

[0899] The server checks the received user ID and password against the database.

[0900] Input: Encrypted user ID and password

[0901] Output: Authentication token

[0902] The server decodes the received user ID and password and checks them against the information stored in the database. If they match, the server generates a JWT (JSON Web Token).

[0903] Step 3:

[0904] The server sends the generated authentication token to the terminal.

[0905] Input: Authentication Token

[0906] Output: Authentication token sent to the device

[0907] The server sends the generated JWT to the device, which stores it and uses it for subsequent authentication.

[0908] Step 4:

[0909] The destination information of the storage device designated by the user is transmitted from the terminal to the server.

[0910] Input: Destination information (e.g., A3 position)

[0911] Output: Destination information sent to the server

[0912] The user uses the application interface to specify the destination of the storage device, and the terminal transmits this information to the server.

[0913] Step 5:

[0914] The server calculates the optimal movement route for the storage device and sends a movement instruction to the storage device.

[0915] Input: Destination information, current storage device location

[0916] Output: Optimal travel route, travel instructions

[0917] The server uses the current location and the specified destination information to execute a route calculation algorithm to calculate the optimal route, and then sends this movement instruction to the storage device.

[0918] Step 6:

[0919] Real-time data on the movement of stored equipment is sent to the server.

[0920] Input: Real-time data of storage equipment

[0921] Output: Real-time data sent to the server

[0922] As the storage equipment begins to move, real-time data from sensors and encoders is collected and sent to a server.

[0923] Step 7:

[0924] The real-time data received from the server is displayed on the terminal.

[0925] Input: Real-time data

[0926] Output: Real-time data displayed on the terminal

[0927] The server organizes the received real-time data and sends it to the terminal, which displays the data to the user in real time.

[0928] Step 8:

[0929] The server periodically checks the status of the safety equipment and, if there is an abnormality, generates an error message and notifies the terminal.

[0930] Input: Safety equipment status data

[0931] Output: Error message, user notification

[0932] The server periodically checks the status of the safety equipment, and if an abnormality is detected, it generates an error message that is immediately sent to the terminal, which then notifies the user.

[0933] Step 9:

[0934] The user receives a notification and takes safety measures.

[0935] Input: Error message

[0936] Output: User's response action

[0937] When the terminal notifies the user of an error message, the user receives it and can stop or change the operation as needed.

[0938] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0939] System configuration

[0940] The system of the present invention is composed of a user, a terminal, a server, and an emotion engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion engine recognizes the user's emotions and adjusts the system's response based on these.

[0941] Program processing overview and specific examples

[0942] User Authentication

[0943] 1. The user enters their ID and password on the device's login screen.

[0944] The user enters their ID and password on the app's login screen and presses the login button.

[0945] 2. The device encrypts the ID and password and sends them to the server.

[0946] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[0947] 3. The server checks the ID and password against the database.

[0948] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[0949] 4. The server generates an authentication token and sends it to the device.

[0950] The server sends the generated authentication token to the terminal, which stores it.

[0951] Determining the location of storage shelves

[0952] 1. The user specifies the destination of the storage shelf on the terminal.

[0953] Users select the location where they want to move the storage shelf using the app's interface.

[0954] 2. The terminal sends the specified destination information to the server.

[0955] The terminal transmits the destination information specified by the user to the server.

[0956] 3. The server calculates the optimal route based on the received information.

[0957] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[0958] 4. The server sends a move instruction to the storage device.

[0959] The server sends a movement instruction to the storage device based on the calculated optimum route.

[0960] Shelf movement and control

[0961] 1. The server receives real-time data from the storage device.

[0962] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[0963] 2. The server sends the received real-time data to the terminal.

[0964] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[0965] 3. The terminal displays the real-time data to the user.

[0966] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[0967] 4. The user changes the operation as needed.

[0968] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[0969] Checking the operation of safety devices

[0970] 1. The server checks the status of the safety device at regular intervals.

[0971] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[0972] 2. The server generates an error message when an abnormality is detected.

[0973] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[0974] 3. The server sends an error message to the terminal.

[0975] The server immediately sends any error messages it generates to the terminal.

[0976] 4. The terminal notifies the user of the error message.

[0977] The device will display the error message to the user using the app's notification function.

[0978] 5. The user receives a notification and takes corrective action.

[0979] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[0980] Implementing the Emotion Engine

[0981] 1. The device recognizes the user's emotions.

[0982] The device's camera, microphone, and other sensors are used to recognize emotions from the user's facial expressions, tone of voice, etc.

[0983] 2. The device sends the emotion data to the emotion engine.

[0984] The device sends the collected emotion data to the emotion engine.

[0985] 3. The emotion engine analyzes the emotion data.

[0986] The emotion engine analyzes the received data to determine the user's current emotional state.

[0987] 4. The server determines the necessary system response based on the emotion engine's analysis results.

[0988] Based on the data from the emotion engine, the server determines an appropriate response, such as simplifying the operating procedure if the user is feeling stressed.

[0989] 5. The server sends the response to the device.

[0990] The server sends an appropriate response to the terminal, which displays the response to the user.

[0991] 6. The user takes action based on the device's response.

[0992] The user can view the response from the terminal and take appropriate action based on it.

[0993] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[0994] The processing flow will be explained below.

[0995] Program processing steps

[0996] User Authentication

[0997] Step 1:

[0998] The user enters their ID and password on the login screen of the device.

[0999] The user enters their ID and password on the app's login screen and presses the login button.

[1000] Step 2:

[1001] The device encrypts the ID and password and sends them to the server.

[1002] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[1003] Step 3:

[1004] The server checks the ID and password against the database.

[1005] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[1006] Step 4:

[1007] The server generates an authentication token and sends it to the device.

[1008] The server sends the generated authentication token to the terminal, which stores it.

[1009] Determining the location of storage shelves

[1010] Step 1:

[1011] The user specifies the destination of the storage shelf on the terminal.

[1012] The user selects a specific location within the app's interface where they want to move the storage shelf, such as "to the left of the living room."

[1013] Step 2:

[1014] The terminal transmits the designated destination information to the server.

[1015] The device sends destination information specified by the user to the server, including the coordinates and direction of the destination.

[1016] Step 3:

[1017] The server calculates the optimal travel route based on the information received.

[1018] The server calculates the optimal route based on the current location of the storage shelf and the specified destination, using the A algorithm or Dijkstra algorithm.

[1019] Step 4:

[1020] The server sends a movement instruction to the storage device.

[1021] The server then sends movement instructions to the storage device based on the calculated optimal route, including start and stop times for the movement, and alternative routes.

[1022] Shelf movement and control

[1023] Step 1:

[1024] A server receives real-time data from the storage device.

[1025] When the storage device starts moving, it sends real-time data from sensors and encoders to the server, including its current position, moving speed, and encoder counts.

[1026] Step 2:

[1027] The server transmits the received real-time data to the terminal.

[1028] The server organizes real-time data received from the storage device and sends it to the user's device, including the device's movement trajectory and current location information.

[1029] Step 3:

[1030] The terminal displays the real-time data to the user.

[1031] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves to the user. For example, the storage shelf icon can be moved in real time on the app.

[1032] Step 4:

[1033] The user changes the operation as needed.

[1034] Users can use the app to change the storage shelf's destination or stop it while it is moving, for example, by issuing commands such as "Stop moving" or "Move to another location."

[1035] Checking the operation of safety devices

[1036] Step 1:

[1037] The server checks the status of the safety device at regular intervals.

[1038] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule, with check intervals set to about 10 seconds.

[1039] Step 2:

[1040] The server generates an error message when an abnormality is detected.

[1041] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected," along with the type and location of the abnormality.

[1042] Step 3:

[1043] The server sends an error message to the terminal.

[1044] The server immediately sends any error messages it generates to the terminal.

[1045] Step 4:

[1046] The terminal notifies the user of the error message.

[1047] The device will display the error message to the user using the app's notification feature, which may include sound and / or vibration.

[1048] Step 5:

[1049] The user receives a notification and takes safety measures.

[1050] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[1051] Implementing the Emotion Engine

[1052] Step 1:

[1053] The device recognizes the user's emotions.

[1054] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice, and sends them to the emotion engine. Evaluation items include smiles, tone of voice, and facial muscle movements.

[1055] Step 2:

[1056] The device transmits the emotion data to the emotion engine.

[1057] The device transmits the collected emotion data to the emotion engine in real time.

[1058] Step 3:

[1059] The emotion engine analyzes the emotion data.

[1060] The emotion engine analyzes the received data with a proprietary algorithm to identify the user's current emotional state, specifically using a machine learning model to classify emotional states such as "stressed" or "relaxed."

[1061] Step 4:

[1062] The server determines the necessary system response based on the analysis results of the emotion engine.

[1063] Based on the results of the emotion engine, the server determines an appropriate response, such as simplifying the operation procedure if the user is feeling stressed.

[1064] Step 5:

[1065] The server sends a response to the terminal.

[1066] The server sends an appropriate response to the terminal, which displays the response to the user.

[1067] Step 6:

[1068] The user takes action according to the terminal's response.

[1069] The user can view the response from the device and take appropriate action based on it, for example by displaying a suggestion such as "Start a relaxing massage."

[1070] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[1071] Example 2

[1072] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1073] Current mobile storage systems have basic functions such as user authentication, moving storage shelves, displaying real-time data, and monitoring safety devices, but they lack system responses that take user emotions into consideration, and there is a need for improved user experience. Another issue is the lack of a mechanism for quickly notifying users when an abnormality is detected. This can lead to complex operations and insufficient safety.

[1074] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1075] In this invention, the server includes means for comparing user authentication information with a database and generating an authentication token, means for calculating an optimal movement route for the storage shelf, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and send the emotion data to an analysis engine, and means for the analysis engine to analyze the emotion data, the server to determine a system response, and display the response on the terminal. This provides an appropriate system response that takes the user's emotion into consideration, making it possible to simplify operation and improve safety.

[1076] "User authentication information" refers to information such as ID and password required for a user to log in to a system.

[1077] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive various data.

[1078] A "server" is a central computer system that connects to a database, performs various calculations and data processing, and sends instructions and information to terminals and other devices.

[1079] An "authentication token" is electronic information that is generated after the server has confirmed the authentication of a user, and is used to maintain the authentication status of the user.

[1080] A "storage shelf" is a movable storage device, which is an item storage device that moves to a specified position based on a user's instructions.

[1081] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[1082] The "optimal movement route" is the most efficient route from the current storage shelf position to the movement destination specified by the user.

[1083] "Real-time data" refers to data that is updated immediately, such as location information and sensor information generated when a storage device moves.

[1084] "Safety devices" are sensors and mechanical protection devices that ensure safety during operation of the storage device.

[1085] An "error message" is a warning or error message that is generated when the system detects an abnormality and is notified to the user.

[1086] "Emotion data" refers to data related to the user's emotions, such as facial expressions and tone of voice, acquired through the device's sensors and camera.

[1087] An "analysis engine" is software or hardware for analyzing emotion data to identify the user's emotional state.

[1088] "System response" refers to the system's actions and display messages that the server determines based on the user's operations and emotional state.

[1089] The system of the present invention is composed of a user, a terminal, a server, and an emotion analysis engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion analysis engine recognizes the user's emotions and adjusts the system's response based on these.

[1090] Hardware and Software Configuration

[1091] Specific examples of system hardware include smartphones and PCs as terminals. These terminals are equipped with cameras, microphones, touchscreens, etc., and provide a means of interface with users. The server uses a high-performance database management system (e.g., MySQL, PostgreSQL), and cloud services can also be used.

[1092] The system's software implementation uses the following technologies:

[1093] Use SSL / TLS encryption (e.g., OpenSSL library) for user authentication.

[1094] RESTful API and WebSocket are used to send and receive data.

[1095] The optimal movement path of the storage shelf is calculated using Dijkstra's algorithm or A algorithm (e.g., NetworkX library).

[1096] Use a JavaScript library (e.g. D3.js) to display real-time data.

[1097] The Python Scikit-learn library is used to check the operation of various safety devices and detect abnormalities.

[1098] OpenCV and Google Cloud Speech API are used to recognize user emotions, and TensorFlow and PyTorch are used to analyze emotion data.

[1099] Example of a system

[1100] Here is a specific example of how the system works. When a user wants to move a storage shelf from the "living room" to the "office," they perform the following operations:

[1101] 1. The user selects the "Office" area on the smartphone app interface and taps the Go button.

[1102] 2. The device sends the selected destination information to the server as an HTTP POST request.

[1103] 3. The server uses the NetworkX library to calculate the optimal route using Dijkstra's algorithm and sends movement instructions to the storage device using the MQTT protocol.

[1104] 4. When the storage device starts moving, it sends real-time data (location information and sensor information) to the server via the MQTT protocol.

[1105] 5. The server organizes the received real-time data and sends it to the user's device using WebSocket.

[1106] 6. The device uses D3.js to visualize the received data on the app screen, showing the user the current location and status of the storage shelf.

[1107] 7. If the user feels tired or stressed while traveling, the device's camera and microphone will collect emotional data and send it to the emotion analysis engine.

[1108] 8. The emotion analysis engine uses TensorFlow to analyze the emotion data and notify the server that the user is feeling stressed.

[1109] 9. Based on the emotion analysis results, the server generates a guide message that simplifies the operation procedure and sends it to the device via WebSocket.

[1110] 10. The user can operate the storage device efficiently by following the guide messages from the terminal and performing the necessary operations.

[1111] Prompt Sentence Examples

[1112] Possible prompts for a generative AI model include:

[1113] "How can I design an application that automatically adjusts its system response based on user emotion?"

[1114] "Please give us a concrete example of an algorithm that uses an emotion analysis engine to detect a user's stress level and simplify the operation procedure."

[1115] This allows the user to safely and efficiently operate the movable storage device while receiving appropriate support according to their emotional state.

[1116] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1117] Step 1:

[1118] The user enters their ID and password on the login screen of the device.

[1119] Input: ID and password entered by the user

[1120] How it works: The user enters their ID and password on the login screen of their smartphone or computer and clicks the login button.

[1121] Output: Device encrypted credentials

[1122] Step 2:

[1123] The device encrypts the entered ID and password and sends them to the server.

[1124] Input: ID and password entered by the user

[1125] How it works: The device encrypts the ID and password using the SSL / TLS protocol (e.g., using the OpenSSL library) and sends it to the server as an HTTP POST request.

[1126] Output: Encrypted credentials sent to the server

[1127] Step 3:

[1128] The server checks the ID and password against the database.

[1129] Input: Encrypted credentials

[1130] What it does: The server accesses a database (e.g. MySQL or PostgreSQL) and runs a SQL query to verify that the entered ID and password match.

[1131] Output: Authentication result (success or failure)

[1132] Step 4:

[1133] The server generates an authentication token and sends it to the device.

[1134] Input: If authentication result is successful

[1135] How it works: The server generates an authentication token using a JWT (JSON Web Token) library and sends it to the device as an HTTP response. The device stores this token in local storage.

[1136] Output: Authentication token

[1137] Step 5:

[1138] The user specifies the destination of the storage shelf on the terminal.

[1139] Input: User-selected destination information

[1140] How it works: The user selects the new location for the storage shelf in the in-app interface and clicks a button to confirm the move.

[1141] Output: Destination information

[1142] Step 6:

[1143] The terminal transmits the designated destination information to the server.

[1144] Input: Destination information

[1145] Operation: The device sends the specified destination information in JSON format to the server. This is done using a RESTful API as an HTTP POST request.

[1146] Output: Destination information is sent to the server

[1147] Step 7:

[1148] The server calculates the optimal travel route based on the information received.

[1149] Input: Current location and specified destination information

[1150] How it works: The server uses Dijkstra's algorithm or A algorithm (using Python's NetworkX library, for example) to calculate the optimal route from the current location to the specified destination.

[1151] Output: Optimal route information

[1152] Step 8:

[1153] The server sends a movement instruction to the storage device.

[1154] Input: Optimal route information

[1155] Operation: The server sends the calculated optimal route information to the storage device. This is done using the MQTT protocol.

[1156] Output: Movement instructions are sent to the storage device

[1157] Step 9:

[1158] Real-time data is sent to the server as the storage device moves.

[1159] Input: Movement instructions

[1160] Operation: The storage device sends position and status information obtained from motors and sensors to a server. MQTT is used as the communication protocol.

[1161] Output: Real-time data

[1162] Step 10:

[1163] The real-time data received by the server is displayed on the terminal.

[1164] Input: Real-time data

[1165] Operation: The server organizes the received data and sends it to the device in real time using WebSocket.

[1166] Output: Organized real-time data

[1167] Step 11:

[1168] The terminal displays the real-time data to the user.

[1169] Input: Organized real-time data

[1170] Operation: The device uses the JavaScript D3.js library to display the received real-time data as graphs and maps on the app screen.

[1171] Output: Visualized current shelf location and status

[1172] Step 12:

[1173] The user changes the operation as needed.

[1174] Input: Real-time data and current operating status

[1175] How it works: The user can change the position and speed of the storage shelf through the app interface. The changed instructions are immediately sent to the server.

[1176] Output: Modified operating instructions

[1177] Step 13:

[1178] The server periodically checks the status of the safety device.

[1179] Input: Safety device sensor information

[1180] How it works: The server periodically schedules tasks to obtain information from safety sensors, for example, using Crontab or a system daemon.

[1181] Output: Safety device status data

[1182] Step 14:

[1183] The server generates an error message when an abnormality is detected.

[1184] Input: Safety device status data

[1185] How it works: The server uses Python's Scikit-learn library to perform anomaly detection and generates an error message if an anomaly is detected.

[1186] Output: Error message

[1187] Step 15:

[1188] The server sends an error message to the terminal.

[1189] Input: Error message

[1190] How it works: The server uses a RESTful API to send an error message to the device.

[1191] Output: Error messages are sent to the terminal

[1192] Step 16:

[1193] The terminal notifies the user of the error message.

[1194] Input: Error message

[1195] What happens: The device sends a push notification to the user using Firebase Cloud Messaging (FCM).

[1196] Output: Error message to the user

[1197] Step 17:

[1198] The user receives a notification and takes safety measures.

[1199] Input: Error message

[1200] Action: The user checks the error message and takes the necessary action (for example, restarting the storage device or performing manual operations).

[1201] Output: Execution of safety instructions

[1202] Step 18:

[1203] The device recognizes the user's emotions.

[1204] Input: User's face and voice data

[1205] How it works: The device uses a camera and microphone to collect user emotions in real time, using OpenCV and Google Cloud Speech API.

[1206] Output: Emotion data

[1207] Step 19:

[1208] The device sends the emotion data to the emotion analysis engine.

[1209] Input: Emotion data

[1210] Operation: The device sends the collected emotion data to the analysis engine via a RESTful API.

[1211] Output: Emotion data is sent to the analysis engine

[1212] Step 20:

[1213] The sentiment analysis engine analyzes the sentiment data.

[1214] Input: Emotion data

[1215] How it works: The sentiment analysis engine uses TensorFlow and PyTorch to analyze emotion data and identify the user's emotional state.

[1216] Output: Emotional state data

[1217] Step 21:

[1218] The server determines the necessary system response based on the emotion analysis results.

[1219] Input: Emotional state data

[1220] Action: The server determines the system response (e.g., simplifying the operation procedure or changing the message) based on the results of the sentiment analysis.

[1221] Output: System response

[1222] Step 22:

[1223] The server sends a response to the terminal.

[1224] Input: System Response

[1225] Operation: The server sends the determined response to the device via WebSocket.

[1226] Output: System responses are sent to the terminal

[1227] Step 23:

[1228] The user takes action according to the terminal's response.

[1229] Input: System Response

[1230] Action: The user checks the response from the device and takes appropriate action based on it. For example, they can simplify the operation by following the guide message.

[1231] Output: Taking appropriate action

[1232] (Application example 2)

[1233] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1234] Conventional storage equipment control systems have issues with operability and convenience because they cannot respond to user operations in real time or take into account the user's stress or emotional state. Furthermore, because the system cannot respond flexibly to emotions, it is unable to respond adequately when the user feels stressed.

[1235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1236] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage shelf destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage shelf and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and transmit the emotion data to an emotion analysis device, and means for the emotion analysis device to analyze the emotion data and adjust the system response based on the analysis result, thereby enabling a flexible system response according to the user's emotional state.

[1237] "User authentication information" refers to information used to authenticate a user to a server, such as a user ID and password.

[1238] The "terminal means" is a device operated by a user and has functions for performing various operations such as transmitting authentication information and specifying the destination of a storage shelf.

[1239] An "authentication token" is a temporary identifier for identifying a user that is generated by a server after the server has successfully authenticated the user.

[1240] A "storage shelf" is a shelf for storing items, and has a structure that allows it to be moved under the control of the system.

[1241] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[1242] "Emotion recognition" refers to a device analyzing a user's facial expressions, tone of voice, etc. to identify the user's current emotional state.

[1243] "Emotion data" is data relating to the user's emotional state obtained through emotion recognition.

[1244] An "emotion analysis device" is a device that analyzes emotion data and identifies the user's emotional state.

[1245] "System response" refers to the reaction or feedback provided by the system in response to the user's actions or emotional state.

[1246] "Real-time data" refers to data relating to the current location and status of a storage device that is acquired when the storage device is moving.

[1247] "Safety devices" are sensors and control devices that detect obstacles and abnormalities while the storage device is in operation and ensure safety.

[1248] An "error message" is a message that is displayed to the user when the system detects an abnormality.

[1249] The present invention is directed to realizing an in-vehicle equipment management application for an autonomous vehicle, and provides flexible system responses according to the user's emotional state. Specific embodiments of the present invention will be described below.

[1250] System configuration

[1251] The system consists of a terminal operated by the user using a smartphone, a server inside the vehicle, a storage device, and an emotion analysis device. The terminal runs an application for remotely controlling the storage shelves inside the vehicle and also has the ability to recognize the user's emotions.

[1252] Program processing overview and specific examples

[1253] 1. User Authentication

[1254] The user enters their ID and password into the smartphone application and sends them to the server. The server compares the encrypted authentication information with a database, and if authentication is successful, it generates a token and sends it to the terminal. Using this token, the user can continue operating the vehicle.

[1255] 2. Storage device control

[1256] The user specifies the destination of the storage shelf through the application interface. The specified location information is sent to the server, which calculates the optimal movement route and sends the instructions to the storage device. The storage device then sends its location information to the server in real time, and the device displays this information to the user, allowing the user to check the current status of the storage shelf.

[1257] 3. Check safety equipment

[1258] The server periodically checks the status of the safety devices of the storage device, and if an abnormality is detected, it generates an error message and sends it to the terminal. The user receives the notification and can correct or stop the operation as necessary.

[1259] 4. Emotion Recognition and Response

[1260] The system recognizes the user's emotions using the device's camera and microphone. Emotional data is sent to an emotion analysis device, and the system's response is adjusted based on the analysis results. For example, if the user is feeling stressed, the system will simplify the operation procedure and change the in-car music and lighting to a more relaxing environment.

[1261] Hardware and software used

[1262] Hardware

[1263] Smartphone (camera, microphone, internet connection)

[1264] In-vehicle server (data processing and control)

[1265] Storage equipment (mechanical equipment for physically moving items)

[1266] Sentiment analyzer (using a sentiment analysis library like EmotionRecognition)

[1267] software

[1268] Authentication library (encryption and authentication token generation)

[1269] Emotion Analysis Library (EmotionRecognition)

[1270] Specific examples

[1271] For example, suppose a user feels very stressed after finishing an important meeting in the car. The system uses the device's camera and microphone to analyze the user's emotions, and if it determines that the user is stressed, it will automatically dim the lights and play relaxing music. At the same time, when the user operates the storage shelves in the car, the system automatically simplifies each operation to ensure smooth operation.

[1272] Prompt Sentence Examples

[1273] "We will build an application that recognizes user emotions. We will also implement a function to adjust the in-car environment based on the user's emotions. For example, if the user is feeling stressed, we will set relaxing music and lighting."

[1274] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1275] Step 1:

[1276] The user launches the application on their smartphone and enters their ID and password.

[1277] Input: User ID and password

[1278] Processing: The terminal encrypts the ID and password using SSL / TLS.

[1279] Output: Encrypted authentication data

[1280] Step 2:

[1281] The terminal transmits the encrypted authentication data to the server.

[1282] Input: Encrypted authentication data

[1283] Processing: The device sends data to the server's authentication API endpoint.

[1284] Output: Authentication request to the server

[1285] Step 3:

[1286] The server compares the received authentication data with a database and generates an authentication token.

[1287] Input: Encrypted authentication data

[1288] Processing: The server searches the database for the corresponding user information, and if a match is found, generates an authentication token.

[1289] Output: Authentication token

[1290] Step 4:

[1291] The server sends the generated authentication token to the terminal.

[1292] Input: Authentication Token

[1293] Processing: The server returns the authentication token to the terminal as an HTTP response.

[1294] Output: Authentication token received on the device

[1295] Step 5:

[1296] The user specifies the destination of the storage shelf in the application.

[1297] Input: Destination information (location coordinates, etc.)

[1298] Process: The user enters the destination in the application interface and presses the send button.

[1299] Output: Destination information data

[1300] Step 6:

[1301] The terminal transmits destination information data to the server.

[1302] Input: Destination information data

[1303] Processing: The terminal sends destination information to the server.

[1304] Output: Send destination information to the server

[1305] Step 7:

[1306] The server calculates the optimal movement route based on the current location of the storage shelf and the specified destination.

[1307] Input: Current location and destination information

[1308] Processing: The server calculates the optimal travel route using a route calculation algorithm.

[1309] Output: Optimal travel route data

[1310] Step 8:

[1311] The server transmits a movement instruction to the storage device based on the optimum movement route.

[1312] Input: Optimal travel route data

[1313] Processing: The server sends a move instruction to the control API of the storage device.

[1314] Output: Movement instructions to storage device

[1315] Step 9:

[1316] The storage device begins to move and transmits its location data to the server in real time.

[1317] Input: Storage device location data

[1318] Processing: The storage device sends the data acquired from the sensors and encoders to the server.

[1319] Output: Sending real-time data to a server

[1320] Step 10:

[1321] The server organizes the received real-time data and sends it to the terminal.

[1322] Input: Real-time location data

[1323] Processing: The server analyzes and organizes the data and sends it to the device as updated information.

[1324] Output: Sending real-time data to the terminal

[1325] Step 11:

[1326] The terminal displays the received real-time data to the user.

[1327] Input: Real-time location data

[1328] Processing: The device displays real-time data in the application UI.

[1329] Output: Display current location to user

[1330] Step 12:

[1331] The terminal recognizes the user's emotions and transmits the emotion data to an emotion analysis device.

[1332] Input: Emotional data such as the user's facial expressions and tone of voice

[1333] Processing: Emotion data is collected using the device's camera and microphone and sent to an emotion analyzer.

[1334] Output: Sending emotional data to the emotion analyzer

[1335] Step 13:

[1336] An emotion analyzer analyzes the emotion data and adjusts the system response based on the analysis results.

[1337] Input: Emotion data

[1338] Processing: The emotion analyzer analyzes the collected data to identify the user's emotional state.

[1339] Output: Instructions for adjusting system response

[1340] Step 14:

[1341] The server adjusts the system response and sends the adjustment result to the terminal.

[1342] Input: Instructions for adjusting system response

[1343] Processing: Based on the results of emotion analysis, the server performs actions such as simplifying operation procedures or changing the in-car environment (music, lighting, etc.).

[1344] Output: Tuned system response

[1345] Step 15:

[1346] The terminal presents the adjusted system response to the user.

[1347] Input: Tuned system response

[1348] Processing: The terminal presents the adjusted system response to the user through the application.

[1349] Output: Display of tuned system response to user

[1350] This allows the user to safely and efficiently operate the storage device while receiving appropriate support according to their emotional state.

[1351] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1352] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1353] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1354] [Third embodiment]

[1355] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1356] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1357] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1358] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1359] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1360] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1361] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1362] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1363] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1364] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1365] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1366] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1367] System configuration

[1368] The system of the present invention comprises a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and set up the storage device.

[1369] Program processing overview and specific examples

[1370] User Authentication

[1371] 1. The user enters their ID and password on the terminal.

[1372] The user enters their ID and password on the app's login screen and presses the login button.

[1373] 2. The device encrypts the user's input and sends it to the server.

[1374] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[1375] 3. The server checks the received ID and password against the database.

[1376] The server searches the database for the corresponding user information and, if there is a match, generates an authentication token.

[1377] 4. The server sends the authentication token to the device.

[1378] The server sends the generated authentication token to the terminal, which stores it.

[1379] Determining the location of storage shelves

[1380] 1. The user specifies the destination of the storage shelf on the terminal.

[1381] Users use the in-app interface to select where they want to move the storage shelf.

[1382] 2. The device sends destination information to the server.

[1383] The terminal sends the specified destination information to the server.

[1384] 3. The server calculates the optimal route based on the received information.

[1385] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[1386] 4. The server sends a move instruction to the storage device.

[1387] The server sends a movement instruction to the storage device according to the calculated movement route.

[1388] Shelf movement and control

[1389] 1. The server receives real-time data from the storage device.

[1390] When the storage device starts moving, real-time data from sensors and encoders is sent to the server.

[1391] 2. The server sends real-time data to the device.

[1392] The server organizes the received real-time data and sends it to the terminal.

[1393] 3. The terminal displays the current status to the user.

[1394] The terminal displays real-time data received from the server, showing the user the current location and status of the storage shelves.

[1395] 4. The user changes the operation as needed.

[1396] Users can use the app to change their destination or instructions while on the move.

[1397] Checking the operation of safety devices

[1398] 1. The server checks the status of the safety device at regular intervals.

[1399] The server periodically checks the status of sensors and safety devices in the storage device.

[1400] 2. The server generates an error message when an abnormality is detected.

[1401] If the server detects an abnormality, it generates an error message such as "Obstacle detected."

[1402] 3. The server sends an error message to the terminal.

[1403] The server immediately sends any error messages it generates to the terminal.

[1404] 4. The terminal notifies the user of the error message.

[1405] When the device receives an error message, it will notify the user using the app's notification function.

[1406] 5. The user receives a notification and takes corrective action.

[1407] The user receives a notification and can stop the operation or check safety if necessary.

[1408] This system allows users to efficiently and safely store items in limited spaces. Its horizontally and vertically movable shelves and lightweight materials make it highly convenient. Its high load-bearing capacity and customizable modular structure allow it to meet a variety of needs. Integrated safety devices and anomaly detection mechanisms ensure peace of mind.

[1409] The processing flow will be explained below.

[1410] Program processing steps

[1411] User Authentication

[1412] Step 1:

[1413] The user enters their ID and password on the login screen of the device.

[1414] The user launches the app and enters their ID and password on the login screen.

[1415] Step 2:

[1416] The device encrypts the ID and password and sends them to the server.

[1417] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[1418] Step 3:

[1419] The server checks the ID and password against the database.

[1420] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[1421] Step 4:

[1422] The server generates an authentication token and sends it to the device.

[1423] The server sends the generated authentication token to the terminal, which stores it.

[1424] Determining the location of storage shelves

[1425] Step 1:

[1426] The user specifies the destination of the storage shelf on the terminal.

[1427] Users select the location where they want to move the storage shelf using the app's interface.

[1428] Step 2:

[1429] The terminal transmits the designated destination information to the server.

[1430] The terminal transmits the destination information specified by the user to the server.

[1431] Step 3:

[1432] The server calculates the optimal travel route based on the information received.

[1433] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[1434] Step 4:

[1435] The server sends a movement instruction to the storage device.

[1436] The server sends a movement instruction to the storage device based on the calculated optimum route.

[1437] Shelf movement and control

[1438] Step 1:

[1439] A server receives real-time data from the storage device.

[1440] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[1441] Step 2:

[1442] The server transmits the received real-time data to the terminal.

[1443] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[1444] Step 3:

[1445] The terminal displays the real-time data to the user.

[1446] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[1447] Step 4:

[1448] The user changes the operation as needed.

[1449] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[1450] Checking the operation of safety devices

[1451] Step 1:

[1452] The server checks the status of the safety device at regular intervals.

[1453] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[1454] Step 2:

[1455] The server generates an error message when an abnormality is detected.

[1456] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[1457] Step 3:

[1458] The server sends an error message to the terminal.

[1459] The server immediately sends any error messages it generates to the terminal.

[1460] Step 4:

[1461] The terminal notifies the user of the error message.

[1462] The device will display the error message to the user using the app's notification function.

[1463] Step 5:

[1464] The user receives a notification and takes safety measures.

[1465] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[1466] This allows the user to operate the mobile storage device efficiently and safely.

[1467] Example 1

[1468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1469] Conventional storage equipment systems have had issues with safety and efficiency. Unauthorized access, breakdowns, and malfunctions are likely to occur during processes such as user authentication, moving storage shelves by position specification, real-time status display, and safety device operation confirmation, making it difficult to provide a system that users can use with confidence. Therefore, there is a demand for a system that allows users to operate storage equipment easily and safely.

[1470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1471] In this invention, the server includes a terminal means for receiving user authentication information and transmitting it to the server, a means for the terminal to encrypt the user's input and transmit it to the server, and a means for the server to compare the received user authentication information with a database and generate an authentication token, thereby preventing unauthorized access and providing a secure authentication process.

[1472] "User authentication information" refers to information such as an ID and password that a user enters to access a system.

[1473] A "server" is a computer system that collates user authentication information, generates authentication tokens, and controls storage devices.

[1474] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive authentication information and destination information.

[1475] A "database" is a collection of data that a server uses to verify user authentication information.

[1476] An "authentication token" is a temporary digital certificate that indicates that a user has been properly authenticated.

[1477] A "storage shelf" is a device that moves to a specified position to store or retrieve items.

[1478] "Destination information" is location information of the destination of the storage shelf designated by the user.

[1479] "Real-time data" is data that is updated immediately to indicate the current location and status of a storage shelf.

[1480] A "safety device" is a device that detects obstacles and stops the storage shelf safely while it is in operation.

[1481] An "error message" is a warning notification that is generated when a safety device detects an abnormality.

[1482] An "algorithm" is a series of calculation procedures that the server uses to calculate the optimal movement path of the storage shelf.

[1483] The "Dijkstra algorithm" is an algorithm based on graph theory that the server uses to calculate the shortest path through the storage shelves.

[1484] The system of the present invention is composed of a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and configure the storage device, thereby enabling safe and efficient storage and retrieval of items.

[1485] First, the user starts the authentication process through the device application. The user enters their ID and password and presses the login button. The device encrypts the entered authentication information using the SSL / TLS protocol and sends it to the server. The server compares the received authentication information with its internal database and, if a match is found, generates an authentication token. This authentication token is sent to the device and stored on the device.

[1486] After authentication, the user can use the application interface to specify the destination of the storage shelf. The specified destination information is sent from the terminal to the server, and the server calculates the optimal movement route. This calculation uses the Dijkstra algorithm to calculate the shortest route that takes into account obstacles between the current storage shelf position and the specified destination. The server then sends the calculated route instructions to the storage device, and the storage device begins moving along the specified route.

[1487] During movement, real-time data from the storage device's sensors and encoders is sent to the server. The server organizes the received data and sends it to the terminal via the WebSocket protocol. The terminal displays the data in real time, showing the user the current location and status of the storage shelf. If necessary, the user can change the movement destination or instructions.

[1488] Furthermore, the operation of safety devices is checked periodically. The server checks data from the storage device's sensors at regular intervals to confirm the status of the safety devices. If an abnormality is detected, the server generates an error message such as "Obstacle detected" and immediately notifies the terminal. After receiving the notification, the user can take the necessary safety measures.

[1489] As a concrete example, if a user wants to move a storage shelf to "Position A," the device sends the information "Destination: Position A" to the server, and the server calculates the optimal route using the Dijkstra algorithm. Also, if an obstacle is detected while the storage shelf is moving, the sensor sends the data "Obstacle detected: Position X, Y" to the server, and an error message is generated.

[1490] Example prompt for a generative AI model:

[1491] "Please explain the SSL / TLS encryption used for login authentication. Please provide a detailed explanation including specific implementation examples."

[1492] The system allows users to efficiently and safely store and retrieve items in limited spaces, and its high load-bearing capacity and customizable modular structure make it suitable for a variety of needs.

[1493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1494] Step 1: The user enters their ID and password.

[1495] The user opens the application on the terminal and enters their ID and password. The input is made into the text boxes on the login screen. The ID (e.g., "user123") and password (e.g., "password123") are recorded on the terminal as input data.

[1496] Step 2: The device encrypts the input and sends it to the server.

[1497] The terminal encrypts the ID and password entered by the user using the SSL / TLS protocol. The encrypted data is sent to the server using an HTTP POST request. At this time, the input data, ID and password, are converted into encrypted form.

[1498] Step 3: The server checks the database.

[1499] The server decrypts the received encrypted data and compares it with its internal database. Specifically, it searches the database for the corresponding user information and checks whether it matches the entered ID and password. If the comparison is successful, it sets a flag indicating "authentication successful."

[1500] Step 4: The server generates and sends an authentication token.

[1501] The server generates an authentication token based on the matching result. This authentication token is in JWT (JSON Web Token) format and contains the user's authentication information. The generated token is sent from the server to the terminal as an HTTP response, which notifies the user that authentication was successful.

[1502] Step 5: The user specifies the movement position.

[1503] The user uses the terminal application interface to specify the destination of the storage shelf. The "destination location information" (e.g., "Location A") is recorded as input data on the terminal. This information is treated as a movement command.

[1504] Step 6: The terminal sends the destination information to the server.

[1505] The terminal sends the specified destination information to the server using an HTTP POST request. The "destination information" as input data is sent to the server and serves as a trigger to start processing.

[1506] Step 7: The server calculates the optimal route.

[1507] The server calculates the optimal route based on the current storage shelf location and the specified destination. Specifically, it calculates the shortest route using the Dijkstra algorithm. The "current location" and "destination location" are processed as input data, and the "optimal route" is obtained as output.

[1508] Step 8: The server sends a transfer instruction to the storage device.

[1509] The server formats the calculated optimal route and sends a movement instruction to the storage device. The route information is sent to the storage device using an HTTP POST request, which causes the storage device to start moving.

[1510] Step 9: The server receives the real-time data.

[1511] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The "current location information" is sent to the server in real time as input data. This allows the current location of the storage shelf to be tracked.

[1512] Step 10: The server sends the data to the terminal.

[1513] The server organizes the received real-time data and sends it to the terminal using the WebSocket protocol. The "organized real-time data" is output to the terminal as input data, allowing the terminal to display the current status.

[1514] Step 11: The device will display the current status.

[1515] The terminal displays the received real-time data in a graphical user interface. The "real-time data" input is drawn on the screen, showing the user the current location and status of the storage shelves.

[1516] Step 12: The user changes the operation.

[1517] The user can check the location of the storage shelf during movement using the app and change the destination or instructions as necessary. The "new destination information" is recorded as input data on the device and then sent back to the server.

[1518] Step 13: The server checks the status.

[1519] The server periodically checks the data sent from the storage device's sensors to confirm the status of the safety devices. The "sensor information" is taken in as input data and a judgment is made as to whether the device is normal or abnormal.

[1520] Step 14: The server generates an error message.

[1521] If abnormal data is received from the sensor, the server generates an error message. The "abnormal information" is processed as input data, and an "error message" is created as output data. Specifically, a message such as "Obstacle detected" is generated.

[1522] Step 15: The server sends an error message to the terminal.

[1523] The server immediately sends the generated error message to the terminal. The "error message" is sent to the terminal as input data and notified in real time.

[1524] Step 16: The terminal notifies the user.

[1525] The terminal notifies the user of the error message by displaying a pop-up message, etc. The "error message" is displayed as input data, and the user is informed of the abnormality.

[1526] Step 17: The user takes safety measures.

[1527] After receiving the notification, the user can take the necessary safety measures, such as stopping the movement of the storage shelf in the app. This action allows the problem to be safely resolved.

[1528] (Application example 1)

[1529] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1530] In logistics centers, to improve the efficiency of storing and retrieving goods, it is necessary to quickly and accurately control the position of storage shelves and storage equipment. However, conventional systems lack the ability to acquire real-time data and check the operation of safety devices, making it difficult to improve efficiency and safety. In addition, there is a need to ensure security and optimality in user authentication and route calculation.

[1531] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1532] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage device destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage device and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and generate an error message and notify the terminal if an abnormality is detected, means for using a JSON web token for login, and means for the server to calculate a movement route for the storage device, including a route calculation algorithm when transmitting instructions to the robot to the system, and notifying the user of an error message in real time as a safety measure. This enables efficient storage management through real-time data acquisition and highly safe operation.

[1533] "User authentication information" refers to information used by a user when accessing a system, and includes, for example, a user ID and password.

[1534] "Terminal means" refers to a device or apparatus that is operated by a user and that communicates with a server to send and receive data.

[1535] A "database" is a structured information collection system for managing and storing data used by an organization or system.

[1536] An "authentication token" is a code that indicates that a user has been authenticated, and is used for session management and access control.

[1537] "Storage equipment" refers to equipment or devices for storing and retrieving items.

[1538] "Destination information" is data relating to the location to which the storage equipment should be moved.

[1539] The "movement route" is the optimum route for the storage equipment to reach the designated destination.

[1540] "Real-time data" is up-to-date information about ongoing processes and conditions.

[1541] "Safety equipment" refers to devices and sensors used to ensure work safety.

[1542] An "error message" is a notification message that is generated when the system detects an abnormality or error.

[1543] "JSON Web Token" is a compact representation format used for user authentication and secure exchange of information.

[1544] "Storage equipment" means equipment or devices for the safe storage of items.

[1545] A "route calculation algorithm" is a method or calculation means for calculating the optimal route to a specified destination.

[1546] To implement this invention, a system including a server, a terminal, and a storage device is required. A specific embodiment of this system will be described below.

[1547] The server first manages the user authentication process. The user enters their ID and password on the device, which then encrypts them using the SSL / TLS protocol and sends them to the server. The server compares the received ID and password with its database, and if they match, generates a JWT (JSON Web Token) and sends it to the device. This achieves secure authentication.

[1548] Next, the destination information of the storage device specified by the user is sent from the terminal to the server. The server uses a route calculation algorithm to calculate the optimal route based on the current location of the storage device and the specified destination information. The server sends this route information to the storage device and issues a movement instruction.

[1549] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The server organizes this data and sends it to the terminal, which then displays the movement status to the user in real time. The user can change the movement destination or instructions as needed.

[1550] To enhance safety, the server periodically checks the status of safety devices. If an abnormality is detected, the server immediately generates an error message and notifies the terminal. The terminal receives this and notifies the user. The user receives the notification and can take safety measures.

[1551] For example, when a user requests on a terminal that "I want to move the shelf to position A3," the server calculates the optimal movement path and sends a movement command to the robot. If an obstacle is detected during movement, an error message is immediately generated and notified to the user.

[1552] Example prompt sentence:

[1553] "The user specified that they wanted the shelf to move to the A3 position."

[1554] This system enables efficient storage management and highly safe operation through real-time data acquisition.

[1555] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1556] Step 1:

[1557] The user enters their ID and password on the device.

[1558] Input: User ID, Password

[1559] Output: Encrypted user ID and password

[1560] The terminal encrypts the ID and password entered by the user on the login screen using the SSL / TLS protocol and sends it to the server in a secure format.

[1561] Step 2:

[1562] The server checks the received user ID and password against the database.

[1563] Input: Encrypted user ID and password

[1564] Output: Authentication token

[1565] The server decodes the received user ID and password and checks them against the information stored in the database. If they match, the server generates a JWT (JSON Web Token).

[1566] Step 3:

[1567] The server sends the generated authentication token to the terminal.

[1568] Input: Authentication Token

[1569] Output: Authentication token sent to the device

[1570] The server sends the generated JWT to the device, which stores it and uses it for subsequent authentication.

[1571] Step 4:

[1572] The destination information of the storage device designated by the user is transmitted from the terminal to the server.

[1573] Input: Destination information (e.g., A3 position)

[1574] Output: Destination information sent to the server

[1575] The user uses the application interface to specify the destination of the storage device, and the terminal transmits this information to the server.

[1576] Step 5:

[1577] The server calculates the optimal movement route for the storage device and sends a movement instruction to the storage device.

[1578] Input: Destination information, current storage device location

[1579] Output: Optimal travel route, travel instructions

[1580] The server uses the current location and the specified destination information to execute a route calculation algorithm to calculate the optimal route, and then sends this movement instruction to the storage device.

[1581] Step 6:

[1582] Real-time data on the movement of stored equipment is sent to the server.

[1583] Input: Real-time data of storage equipment

[1584] Output: Real-time data sent to the server

[1585] As the storage equipment begins to move, real-time data from sensors and encoders is collected and sent to a server.

[1586] Step 7:

[1587] The real-time data received from the server is displayed on the terminal.

[1588] Input: Real-time data

[1589] Output: Real-time data displayed on the terminal

[1590] The server organizes the received real-time data and sends it to the terminal, which displays the data to the user in real time.

[1591] Step 8:

[1592] The server periodically checks the status of the safety equipment and, if there is an abnormality, generates an error message and notifies the terminal.

[1593] Input: Safety equipment status data

[1594] Output: Error message, user notification

[1595] The server periodically checks the status of the safety equipment, and if an abnormality is detected, it generates an error message that is immediately sent to the terminal, which then notifies the user.

[1596] Step 9:

[1597] The user receives a notification and takes safety measures.

[1598] Input: Error message

[1599] Output: User's response action

[1600] When the terminal notifies the user of an error message, the user receives it and can stop or change the operation as needed.

[1601] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1602] System configuration

[1603] The system of the present invention is composed of a user, a terminal, a server, and an emotion engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion engine recognizes the user's emotions and adjusts the system's response based on these.

[1604] Program processing overview and specific examples

[1605] User Authentication

[1606] 1. The user enters their ID and password on the device's login screen.

[1607] The user enters their ID and password on the app's login screen and presses the login button.

[1608] 2. The device encrypts the ID and password and sends them to the server.

[1609] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[1610] 3. The server checks the ID and password against the database.

[1611] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[1612] 4. The server generates an authentication token and sends it to the device.

[1613] The server sends the generated authentication token to the terminal, which stores it.

[1614] Determining the location of storage shelves

[1615] 1. The user specifies the destination of the storage shelf on the terminal.

[1616] Users select the location where they want to move the storage shelf using the app's interface.

[1617] 2. The terminal sends the specified destination information to the server.

[1618] The terminal transmits the destination information specified by the user to the server.

[1619] 3. The server calculates the optimal route based on the received information.

[1620] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[1621] 4. The server sends a move instruction to the storage device.

[1622] The server sends a movement instruction to the storage device based on the calculated optimum route.

[1623] Shelf movement and control

[1624] 1. The server receives real-time data from the storage device.

[1625] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[1626] 2. The server sends the received real-time data to the terminal.

[1627] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[1628] 3. The terminal displays the real-time data to the user.

[1629] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[1630] 4. The user changes the operation as needed.

[1631] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[1632] Checking the operation of safety devices

[1633] 1. The server checks the status of the safety device at regular intervals.

[1634] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[1635] 2. The server generates an error message when an abnormality is detected.

[1636] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[1637] 3. The server sends an error message to the terminal.

[1638] The server immediately sends any error messages it generates to the terminal.

[1639] 4. The terminal notifies the user of the error message.

[1640] The device will display the error message to the user using the app's notification function.

[1641] 5. The user receives a notification and takes corrective action.

[1642] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[1643] Implementing the Emotion Engine

[1644] 1. The device recognizes the user's emotions.

[1645] The device's camera, microphone, and other sensors are used to recognize emotions from the user's facial expressions, tone of voice, etc.

[1646] 2. The device sends the emotion data to the emotion engine.

[1647] The device sends the collected emotion data to the emotion engine.

[1648] 3. The emotion engine analyzes the emotion data.

[1649] The emotion engine analyzes the received data to determine the user's current emotional state.

[1650] 4. The server determines the necessary system response based on the emotion engine's analysis results.

[1651] Based on the data from the emotion engine, the server determines an appropriate response, such as simplifying the operating procedure if the user is feeling stressed.

[1652] 5. The server sends the response to the device.

[1653] The server sends an appropriate response to the terminal, which displays the response to the user.

[1654] 6. The user takes action based on the device's response.

[1655] The user can view the response from the terminal and take appropriate action based on it.

[1656] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[1657] The processing flow will be explained below.

[1658] Program processing steps

[1659] User Authentication

[1660] Step 1:

[1661] The user enters their ID and password on the login screen of the device.

[1662] The user enters their ID and password on the app's login screen and presses the login button.

[1663] Step 2:

[1664] The device encrypts the ID and password and sends them to the server.

[1665] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[1666] Step 3:

[1667] The server checks the ID and password against the database.

[1668] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[1669] Step 4:

[1670] The server generates an authentication token and sends it to the device.

[1671] The server sends the generated authentication token to the terminal, which stores it.

[1672] Determining the location of storage shelves

[1673] Step 1:

[1674] The user specifies the destination of the storage shelf on the terminal.

[1675] The user selects a specific location within the app's interface where they want to move the storage shelf, such as "to the left of the living room."

[1676] Step 2:

[1677] The terminal transmits the designated destination information to the server.

[1678] The device sends destination information specified by the user to the server, including the coordinates and direction of the destination.

[1679] Step 3:

[1680] The server calculates the optimal travel route based on the information received.

[1681] The server calculates the optimal route based on the current location of the storage shelf and the specified destination, using the A algorithm or Dijkstra algorithm.

[1682] Step 4:

[1683] The server sends a movement instruction to the storage device.

[1684] The server then sends movement instructions to the storage device based on the calculated optimal route, including start and stop times for the movement, and alternative routes.

[1685] Shelf movement and control

[1686] Step 1:

[1687] A server receives real-time data from the storage device.

[1688] When the storage device starts moving, it sends real-time data from sensors and encoders to the server, including its current position, moving speed, and encoder counts.

[1689] Step 2:

[1690] The server transmits the received real-time data to the terminal.

[1691] The server organizes real-time data received from the storage device and sends it to the user's device, including the device's movement trajectory and current location information.

[1692] Step 3:

[1693] The terminal displays the real-time data to the user.

[1694] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves to the user. For example, the storage shelf icon can be moved in real time on the app.

[1695] Step 4:

[1696] The user changes the operation as needed.

[1697] Users can use the app to change the storage shelf's destination or stop it while it is moving, for example, by issuing commands such as "Stop moving" or "Move to another location."

[1698] Checking the operation of safety devices

[1699] Step 1:

[1700] The server checks the status of the safety device at regular intervals.

[1701] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule, with check intervals set to about 10 seconds.

[1702] Step 2:

[1703] The server generates an error message when an abnormality is detected.

[1704] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected," along with the type and location of the abnormality.

[1705] Step 3:

[1706] The server sends an error message to the terminal.

[1707] The server immediately sends any error messages it generates to the terminal.

[1708] Step 4:

[1709] The terminal notifies the user of the error message.

[1710] The device will display the error message to the user using the app's notification feature, which may include sound and / or vibration.

[1711] Step 5:

[1712] The user receives a notification and takes safety measures.

[1713] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[1714] Implementing the Emotion Engine

[1715] Step 1:

[1716] The device recognizes the user's emotions.

[1717] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice, and sends them to the emotion engine. Evaluation items include smiles, tone of voice, and facial muscle movements.

[1718] Step 2:

[1719] The device transmits the emotion data to the emotion engine.

[1720] The device transmits the collected emotion data to the emotion engine in real time.

[1721] Step 3:

[1722] The emotion engine analyzes the emotion data.

[1723] The emotion engine analyzes the received data with a proprietary algorithm to identify the user's current emotional state, specifically using a machine learning model to classify emotional states such as "stressed" or "relaxed."

[1724] Step 4:

[1725] The server determines the necessary system response based on the analysis results of the emotion engine.

[1726] Based on the results of the emotion engine, the server determines an appropriate response, such as simplifying the operation procedure if the user is feeling stressed.

[1727] Step 5:

[1728] The server sends a response to the terminal.

[1729] The server sends an appropriate response to the terminal, which displays the response to the user.

[1730] Step 6:

[1731] The user takes action according to the terminal's response.

[1732] The user can view the response from the device and take appropriate action based on it, for example by displaying a suggestion such as "Start a relaxing massage."

[1733] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[1734] Example 2

[1735] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1736] Current mobile storage systems have basic functions such as user authentication, moving storage shelves, displaying real-time data, and monitoring safety devices, but they lack system responses that take user emotions into consideration, and there is a need for improved user experience. Another issue is the lack of a mechanism for quickly notifying users when an abnormality is detected. This can lead to complex operations and insufficient safety.

[1737] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1738] In this invention, the server includes means for comparing user authentication information with a database and generating an authentication token, means for calculating an optimal movement route for the storage shelf, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and send the emotion data to an analysis engine, and means for the analysis engine to analyze the emotion data, the server to determine a system response, and display the response on the terminal. This provides an appropriate system response that takes the user's emotion into consideration, making it possible to simplify operation and improve safety.

[1739] "User authentication information" refers to information such as ID and password required for a user to log in to a system.

[1740] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive various data.

[1741] A "server" is a central computer system that connects to a database, performs various calculations and data processing, and sends instructions and information to terminals and other devices.

[1742] An "authentication token" is electronic information that is generated after the server has confirmed the authentication of a user, and is used to maintain the authentication status of the user.

[1743] A "storage shelf" is a movable storage device, which is an item storage device that moves to a specified position based on a user's instructions.

[1744] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[1745] The "optimal movement route" is the most efficient route from the current storage shelf position to the movement destination specified by the user.

[1746] "Real-time data" refers to data that is updated immediately, such as location information and sensor information generated when a storage device moves.

[1747] "Safety devices" are sensors and mechanical protection devices that ensure safety during operation of the storage device.

[1748] An "error message" is a warning or error message that is generated when the system detects an abnormality and is notified to the user.

[1749] "Emotion data" refers to data related to the user's emotions, such as facial expressions and tone of voice, acquired through the device's sensors and camera.

[1750] An "analysis engine" is software or hardware for analyzing emotion data to identify the user's emotional state.

[1751] "System response" refers to the system's actions and display messages that the server determines based on the user's operations and emotional state.

[1752] The system of the present invention is composed of a user, a terminal, a server, and an emotion analysis engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion analysis engine recognizes the user's emotions and adjusts the system's response based on these.

[1753] Hardware and Software Configuration

[1754] Specific examples of system hardware include smartphones and PCs as terminals. These terminals are equipped with cameras, microphones, touchscreens, etc., and provide a means of interface with users. The server uses a high-performance database management system (e.g., MySQL, PostgreSQL), and cloud services can also be used.

[1755] The system's software implementation uses the following technologies:

[1756] Use SSL / TLS encryption (e.g., OpenSSL library) for user authentication.

[1757] RESTful API and WebSocket are used to send and receive data.

[1758] The optimal movement path of the storage shelf is calculated using Dijkstra's algorithm or A algorithm (e.g., NetworkX library).

[1759] Use a JavaScript library (e.g. D3.js) to display real-time data.

[1760] The Python Scikit-learn library is used to check the operation of various safety devices and detect abnormalities.

[1761] OpenCV and Google Cloud Speech API are used to recognize user emotions, and TensorFlow and PyTorch are used to analyze emotion data.

[1762] Example of a system

[1763] Here is a specific example of how the system works. When a user wants to move a storage shelf from the "living room" to the "office," they perform the following operations:

[1764] 1. The user selects the "Office" area on the smartphone app interface and taps the Go button.

[1765] 2. The device sends the selected destination information to the server as an HTTP POST request.

[1766] 3. The server uses the NetworkX library to calculate the optimal route using Dijkstra's algorithm and sends movement instructions to the storage device using the MQTT protocol.

[1767] 4. When the storage device starts moving, it sends real-time data (location information and sensor information) to the server via the MQTT protocol.

[1768] 5. The server organizes the received real-time data and sends it to the user's device using WebSocket.

[1769] 6. The device uses D3.js to visualize the received data on the app screen, showing the user the current location and status of the storage shelf.

[1770] 7. If the user feels tired or stressed while traveling, the device's camera and microphone will collect emotional data and send it to the emotion analysis engine.

[1771] 8. The emotion analysis engine uses TensorFlow to analyze the emotion data and notify the server that the user is feeling stressed.

[1772] 9. Based on the emotion analysis results, the server generates a guide message that simplifies the operation procedure and sends it to the device via WebSocket.

[1773] 10. The user can operate the storage device efficiently by following the guide messages from the terminal and performing the necessary operations.

[1774] Prompt Sentence Examples

[1775] Possible prompts for a generative AI model include:

[1776] "How can I design an application that automatically adjusts its system response based on user emotion?"

[1777] "Please give us a concrete example of an algorithm that uses an emotion analysis engine to detect a user's stress level and simplify the operation procedure."

[1778] This allows the user to safely and efficiently operate the movable storage device while receiving appropriate support according to their emotional state.

[1779] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1780] Step 1:

[1781] The user enters their ID and password on the login screen of the device.

[1782] Input: ID and password entered by the user

[1783] How it works: The user enters their ID and password on the login screen of their smartphone or computer and clicks the login button.

[1784] Output: Device encrypted credentials

[1785] Step 2:

[1786] The device encrypts the entered ID and password and sends them to the server.

[1787] Input: ID and password entered by the user

[1788] How it works: The device encrypts the ID and password using the SSL / TLS protocol (e.g., using the OpenSSL library) and sends it to the server as an HTTP POST request.

[1789] Output: Encrypted credentials sent to the server

[1790] Step 3:

[1791] The server checks the ID and password against the database.

[1792] Input: Encrypted credentials

[1793] What it does: The server accesses a database (e.g. MySQL or PostgreSQL) and runs a SQL query to verify that the entered ID and password match.

[1794] Output: Authentication result (success or failure)

[1795] Step 4:

[1796] The server generates an authentication token and sends it to the device.

[1797] Input: If authentication result is successful

[1798] How it works: The server generates an authentication token using a JWT (JSON Web Token) library and sends it to the device as an HTTP response. The device stores this token in local storage.

[1799] Output: Authentication token

[1800] Step 5:

[1801] The user specifies the destination of the storage shelf on the terminal.

[1802] Input: User-selected destination information

[1803] How it works: The user selects the new location for the storage shelf in the in-app interface and clicks a button to confirm the move.

[1804] Output: Destination information

[1805] Step 6:

[1806] The terminal transmits the designated destination information to the server.

[1807] Input: Destination information

[1808] Operation: The device sends the specified destination information in JSON format to the server. This is done using a RESTful API as an HTTP POST request.

[1809] Output: Destination information is sent to the server

[1810] Step 7:

[1811] The server calculates the optimal travel route based on the information received.

[1812] Input: Current location and specified destination information

[1813] How it works: The server uses Dijkstra's algorithm or A algorithm (using Python's NetworkX library, for example) to calculate the optimal route from the current location to the specified destination.

[1814] Output: Optimal route information

[1815] Step 8:

[1816] The server sends a movement instruction to the storage device.

[1817] Input: Optimal route information

[1818] Operation: The server sends the calculated optimal route information to the storage device. This is done using the MQTT protocol.

[1819] Output: Movement instructions are sent to the storage device

[1820] Step 9:

[1821] Real-time data is sent to the server as the storage device moves.

[1822] Input: Movement instructions

[1823] Operation: The storage device sends position and status information obtained from motors and sensors to a server. MQTT is used as the communication protocol.

[1824] Output: Real-time data

[1825] Step 10:

[1826] The real-time data received by the server is displayed on the terminal.

[1827] Input: Real-time data

[1828] Operation: The server organizes the received data and sends it to the device in real time using WebSocket.

[1829] Output: Organized real-time data

[1830] Step 11:

[1831] The terminal displays the real-time data to the user.

[1832] Input: Organized real-time data

[1833] Operation: The device uses the JavaScript D3.js library to display the received real-time data as graphs and maps on the app screen.

[1834] Output: Visualized current shelf location and status

[1835] Step 12:

[1836] The user changes the operation as needed.

[1837] Input: Real-time data and current operating status

[1838] How it works: The user can change the position and speed of the storage shelf through the app interface. The changed instructions are immediately sent to the server.

[1839] Output: Modified operating instructions

[1840] Step 13:

[1841] The server periodically checks the status of the safety device.

[1842] Input: Safety device sensor information

[1843] How it works: The server periodically schedules tasks to obtain information from safety sensors, for example, using Crontab or a system daemon.

[1844] Output: Safety device status data

[1845] Step 14:

[1846] The server generates an error message when an abnormality is detected.

[1847] Input: Safety device status data

[1848] How it works: The server uses Python's Scikit-learn library to perform anomaly detection and generates an error message if an anomaly is detected.

[1849] Output: Error message

[1850] Step 15:

[1851] The server sends an error message to the terminal.

[1852] Input: Error message

[1853] How it works: The server uses a RESTful API to send an error message to the device.

[1854] Output: Error messages are sent to the terminal

[1855] Step 16:

[1856] The terminal notifies the user of the error message.

[1857] Input: Error message

[1858] What happens: The device sends a push notification to the user using Firebase Cloud Messaging (FCM).

[1859] Output: Error message to the user

[1860] Step 17:

[1861] The user receives a notification and takes safety measures.

[1862] Input: Error message

[1863] Action: The user checks the error message and takes the necessary action (for example, restarting the storage device or performing manual operations).

[1864] Output: Execution of safety instructions

[1865] Step 18:

[1866] The device recognizes the user's emotions.

[1867] Input: User's face and voice data

[1868] How it works: The device uses a camera and microphone to collect user emotions in real time, using OpenCV and Google Cloud Speech API.

[1869] Output: Emotion data

[1870] Step 19:

[1871] The device sends the emotion data to the emotion analysis engine.

[1872] Input: Emotion data

[1873] Operation: The device sends the collected emotion data to the analysis engine via a RESTful API.

[1874] Output: Emotion data is sent to the analysis engine

[1875] Step 20:

[1876] The sentiment analysis engine analyzes the sentiment data.

[1877] Input: Emotion data

[1878] How it works: The sentiment analysis engine uses TensorFlow and PyTorch to analyze emotion data and identify the user's emotional state.

[1879] Output: Emotional state data

[1880] Step 21:

[1881] The server determines the necessary system response based on the emotion analysis results.

[1882] Input: Emotional state data

[1883] Action: The server determines the system response (e.g., simplifying the operation procedure or changing the message) based on the results of the sentiment analysis.

[1884] Output: System response

[1885] Step 22:

[1886] The server sends a response to the terminal.

[1887] Input: System Response

[1888] Operation: The server sends the determined response to the device via WebSocket.

[1889] Output: System responses are sent to the terminal

[1890] Step 23:

[1891] The user takes action according to the terminal's response.

[1892] Input: System Response

[1893] Action: The user checks the response from the device and takes appropriate action based on it. For example, they can simplify the operation by following the guide message.

[1894] Output: Taking appropriate action

[1895] (Application example 2)

[1896] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1897] Conventional storage equipment control systems have issues with operability and convenience because they cannot respond to user operations in real time or take into account the user's stress or emotional state. Furthermore, because the system cannot respond flexibly to emotions, it is unable to respond adequately when the user feels stressed.

[1898] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1899] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage shelf destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage shelf and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and transmit the emotion data to an emotion analysis device, and means for the emotion analysis device to analyze the emotion data and adjust the system response based on the analysis result, thereby enabling a flexible system response according to the user's emotional state.

[1900] "User authentication information" refers to information used to authenticate a user to a server, such as a user ID and password.

[1901] The "terminal means" is a device operated by a user and has functions for performing various operations such as transmitting authentication information and specifying the destination of a storage shelf.

[1902] An "authentication token" is a temporary identifier for identifying a user that is generated by a server after the server has successfully authenticated the user.

[1903] A "storage shelf" is a shelf for storing items, and has a structure that allows it to be moved under the control of the system.

[1904] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[1905] "Emotion recognition" refers to a device analyzing a user's facial expressions, tone of voice, etc. to identify the user's current emotional state.

[1906] "Emotion data" is data relating to the user's emotional state obtained through emotion recognition.

[1907] An "emotion analysis device" is a device that analyzes emotion data and identifies the user's emotional state.

[1908] "System response" refers to the reaction or feedback provided by the system in response to the user's actions or emotional state.

[1909] "Real-time data" refers to data relating to the current location and status of a storage device that is acquired when the storage device is moving.

[1910] "Safety devices" are sensors and control devices that detect obstacles and abnormalities while the storage device is in operation and ensure safety.

[1911] An "error message" is a message that is displayed to the user when the system detects an abnormality.

[1912] The present invention is directed to realizing an in-vehicle equipment management application for an autonomous vehicle, and provides flexible system responses according to the user's emotional state. Specific embodiments of the present invention will be described below.

[1913] System configuration

[1914] The system consists of a terminal operated by the user using a smartphone, a server inside the vehicle, a storage device, and an emotion analysis device. The terminal runs an application for remotely controlling the storage shelves inside the vehicle and also has the ability to recognize the user's emotions.

[1915] Program processing overview and specific examples

[1916] 1. User Authentication

[1917] The user enters their ID and password into the smartphone application and sends them to the server. The server compares the encrypted authentication information with a database, and if authentication is successful, it generates a token and sends it to the terminal. Using this token, the user can continue operating the vehicle.

[1918] 2. Storage device control

[1919] The user specifies the destination of the storage shelf through the application interface. The specified location information is sent to the server, which calculates the optimal movement route and sends the instructions to the storage device. The storage device then sends its location information to the server in real time, and the device displays this information to the user, allowing the user to check the current status of the storage shelf.

[1920] 3. Check safety equipment

[1921] The server periodically checks the status of the safety devices of the storage device, and if an abnormality is detected, it generates an error message and sends it to the terminal. The user receives the notification and can correct or stop the operation as necessary.

[1922] 4. Emotion Recognition and Response

[1923] The system recognizes the user's emotions using the device's camera and microphone. Emotional data is sent to an emotion analysis device, and the system's response is adjusted based on the analysis results. For example, if the user is feeling stressed, the system will simplify the operation procedure and change the in-car music and lighting to a more relaxing environment.

[1924] Hardware and software used

[1925] Hardware

[1926] Smartphone (camera, microphone, internet connection)

[1927] In-vehicle server (data processing and control)

[1928] Storage equipment (mechanical equipment for physically moving items)

[1929] Sentiment analyzer (using a sentiment analysis library like EmotionRecognition)

[1930] software

[1931] Authentication library (encryption and authentication token generation)

[1932] Emotion Analysis Library (EmotionRecognition)

[1933] Specific examples

[1934] For example, suppose a user feels very stressed after finishing an important meeting in the car. The system uses the device's camera and microphone to analyze the user's emotions, and if it determines that the user is stressed, it will automatically dim the lights and play relaxing music. At the same time, when the user operates the storage shelves in the car, the system automatically simplifies each operation to ensure smooth operation.

[1935] Prompt Sentence Examples

[1936] "We will build an application that recognizes user emotions. We will also implement a function to adjust the in-car environment based on the user's emotions. For example, if the user is feeling stressed, we will set relaxing music and lighting."

[1937] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1938] Step 1:

[1939] The user launches the application on their smartphone and enters their ID and password.

[1940] Input: User ID and password

[1941] Processing: The terminal encrypts the ID and password using SSL / TLS.

[1942] Output: Encrypted authentication data

[1943] Step 2:

[1944] The terminal transmits the encrypted authentication data to the server.

[1945] Input: Encrypted authentication data

[1946] Processing: The device sends data to the server's authentication API endpoint.

[1947] Output: Authentication request to the server

[1948] Step 3:

[1949] The server compares the received authentication data with a database and generates an authentication token.

[1950] Input: Encrypted authentication data

[1951] Processing: The server searches the database for the corresponding user information, and if a match is found, generates an authentication token.

[1952] Output: Authentication token

[1953] Step 4:

[1954] The server sends the generated authentication token to the terminal.

[1955] Input: Authentication Token

[1956] Processing: The server returns the authentication token to the terminal as an HTTP response.

[1957] Output: Authentication token received on the device

[1958] Step 5:

[1959] The user specifies the destination of the storage shelf in the application.

[1960] Input: Destination information (location coordinates, etc.)

[1961] Process: The user enters the destination in the application interface and presses the send button.

[1962] Output: Destination information data

[1963] Step 6:

[1964] The terminal transmits destination information data to the server.

[1965] Input: Destination information data

[1966] Processing: The terminal sends destination information to the server.

[1967] Output: Send destination information to the server

[1968] Step 7:

[1969] The server calculates the optimal movement route based on the current location of the storage shelf and the specified destination.

[1970] Input: Current location and destination information

[1971] Processing: The server calculates the optimal travel route using a route calculation algorithm.

[1972] Output: Optimal travel route data

[1973] Step 8:

[1974] The server transmits a movement instruction to the storage device based on the optimum movement route.

[1975] Input: Optimal travel route data

[1976] Processing: The server sends a move instruction to the control API of the storage device.

[1977] Output: Movement instructions to storage device

[1978] Step 9:

[1979] The storage device begins to move and transmits its location data to the server in real time.

[1980] Input: Storage device location data

[1981] Processing: The storage device sends the data acquired from the sensors and encoders to the server.

[1982] Output: Sending real-time data to a server

[1983] Step 10:

[1984] The server organizes the received real-time data and sends it to the terminal.

[1985] Input: Real-time location data

[1986] Processing: The server analyzes and organizes the data and sends it to the device as updated information.

[1987] Output: Sending real-time data to the terminal

[1988] Step 11:

[1989] The terminal displays the received real-time data to the user.

[1990] Input: Real-time location data

[1991] Processing: The device displays real-time data in the application UI.

[1992] Output: Display current location to user

[1993] Step 12:

[1994] The terminal recognizes the user's emotions and transmits the emotion data to an emotion analysis device.

[1995] Input: Emotional data such as the user's facial expressions and tone of voice

[1996] Processing: Emotion data is collected using the device's camera and microphone and sent to an emotion analyzer.

[1997] Output: Sending emotional data to the emotion analyzer

[1998] Step 13:

[1999] An emotion analyzer analyzes the emotion data and adjusts the system response based on the analysis results.

[2000] Input: Emotion data

[2001] Processing: The emotion analyzer analyzes the collected data to identify the user's emotional state.

[2002] Output: Instructions for adjusting system response

[2003] Step 14:

[2004] The server adjusts the system response and sends the adjustment result to the terminal.

[2005] Input: Instructions for adjusting system response

[2006] Processing: Based on the results of emotion analysis, the server performs actions such as simplifying operation procedures or changing the in-car environment (music, lighting, etc.).

[2007] Output: Tuned system response

[2008] Step 15:

[2009] The terminal presents the adjusted system response to the user.

[2010] Input: Tuned system response

[2011] Processing: The terminal presents the adjusted system response to the user through the application.

[2012] Output: Display of tuned system response to user

[2013] This allows the user to safely and efficiently operate the storage device while receiving appropriate support according to their emotional state.

[2014] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2015] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2016] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2017] [Fourth embodiment]

[2018] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2019] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2020] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2021] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2022] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2023] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2024] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2025] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2026] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2027] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2028] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2029] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2030] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2031] System configuration

[2032] The system of the present invention comprises a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and set up the storage device.

[2033] Program processing overview and specific examples

[2034] User Authentication

[2035] 1. The user enters their ID and password on the terminal.

[2036] The user enters their ID and password on the app's login screen and presses the login button.

[2037] 2. The device encrypts the user's input and sends it to the server.

[2038] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[2039] 3. The server checks the received ID and password against the database.

[2040] The server searches the database for the corresponding user information and, if there is a match, generates an authentication token.

[2041] 4. The server sends the authentication token to the device.

[2042] The server sends the generated authentication token to the terminal, which stores it.

[2043] Determining the location of storage shelves

[2044] 1. The user specifies the destination of the storage shelf on the terminal.

[2045] Users use the in-app interface to select where they want to move the storage shelf.

[2046] 2. The device sends destination information to the server.

[2047] The terminal sends the specified destination information to the server.

[2048] 3. The server calculates the optimal route based on the received information.

[2049] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[2050] 4. The server sends a move instruction to the storage device.

[2051] The server sends a movement instruction to the storage device according to the calculated movement route.

[2052] Shelf movement and control

[2053] 1. The server receives real-time data from the storage device.

[2054] When the storage device starts moving, real-time data from sensors and encoders is sent to the server.

[2055] 2. The server sends real-time data to the device.

[2056] The server organizes the received real-time data and sends it to the terminal.

[2057] 3. The terminal displays the current status to the user.

[2058] The terminal displays real-time data received from the server, showing the user the current location and status of the storage shelves.

[2059] 4. The user changes the operation as needed.

[2060] Users can use the app to change their destination or instructions while on the move.

[2061] Checking the operation of safety devices

[2062] 1. The server checks the status of the safety device at regular intervals.

[2063] The server periodically checks the status of sensors and safety devices in the storage device.

[2064] 2. The server generates an error message when an abnormality is detected.

[2065] If the server detects an abnormality, it generates an error message such as "Obstacle detected."

[2066] 3. The server sends an error message to the terminal.

[2067] The server immediately sends any error messages it generates to the terminal.

[2068] 4. The terminal notifies the user of the error message.

[2069] When the device receives an error message, it will notify the user using the app's notification function.

[2070] 5. The user receives a notification and takes corrective action.

[2071] The user receives a notification and can stop the operation or check safety if necessary.

[2072] This system allows users to efficiently and safely store items in limited spaces. Its horizontally and vertically movable shelves and lightweight materials make it highly convenient. Its high load-bearing capacity and customizable modular structure allow it to meet a variety of needs. Integrated safety devices and anomaly detection mechanisms ensure peace of mind.

[2073] The processing flow will be explained below.

[2074] Program processing steps

[2075] User Authentication

[2076] Step 1:

[2077] The user enters their ID and password on the login screen of the device.

[2078] The user launches the app and enters their ID and password on the login screen.

[2079] Step 2:

[2080] The device encrypts the ID and password and sends them to the server.

[2081] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[2082] Step 3:

[2083] The server checks the ID and password against the database.

[2084] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[2085] Step 4:

[2086] The server generates an authentication token and sends it to the device.

[2087] The server sends the generated authentication token to the terminal, which stores it.

[2088] Determining the location of storage shelves

[2089] Step 1:

[2090] The user specifies the destination of the storage shelf on the terminal.

[2091] Users select the location where they want to move the storage shelf using the app's interface.

[2092] Step 2:

[2093] The terminal transmits the designated destination information to the server.

[2094] The terminal transmits the destination information specified by the user to the server.

[2095] Step 3:

[2096] The server calculates the optimal travel route based on the information received.

[2097] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[2098] Step 4:

[2099] The server sends a movement instruction to the storage device.

[2100] The server sends a movement instruction to the storage device based on the calculated optimum route.

[2101] Shelf movement and control

[2102] Step 1:

[2103] A server receives real-time data from the storage device.

[2104] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[2105] Step 2:

[2106] The server transmits the received real-time data to the terminal.

[2107] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[2108] Step 3:

[2109] The terminal displays the real-time data to the user.

[2110] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[2111] Step 4:

[2112] The user changes the operation as needed.

[2113] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[2114] Checking the operation of safety devices

[2115] Step 1:

[2116] The server checks the status of the safety device at regular intervals.

[2117] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[2118] Step 2:

[2119] The server generates an error message when an abnormality is detected.

[2120] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[2121] Step 3:

[2122] The server sends an error message to the terminal.

[2123] The server immediately sends any error messages it generates to the terminal.

[2124] Step 4:

[2125] The terminal notifies the user of the error message.

[2126] The device will display the error message to the user using the app's notification function.

[2127] Step 5:

[2128] The user receives a notification and takes safety measures.

[2129] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[2130] This allows the user to operate the mobile storage device efficiently and safely.

[2131] Example 1

[2132] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2133] Conventional storage equipment systems have had issues with safety and efficiency. Unauthorized access, breakdowns, and malfunctions are likely to occur during processes such as user authentication, moving storage shelves by position specification, real-time status display, and safety device operation confirmation, making it difficult to provide a system that users can use with confidence. Therefore, there is a demand for a system that allows users to operate storage equipment easily and safely.

[2134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2135] In this invention, the server includes a terminal means for receiving user authentication information and transmitting it to the server, a means for the terminal to encrypt the user's input and transmit it to the server, and a means for the server to compare the received user authentication information with a database and generate an authentication token, thereby preventing unauthorized access and providing a secure authentication process.

[2136] "User authentication information" refers to information such as an ID and password that a user enters to access a system.

[2137] A "server" is a computer system that collates user authentication information, generates authentication tokens, and controls storage devices.

[2138] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive authentication information and destination information.

[2139] A "database" is a collection of data that a server uses to verify user authentication information.

[2140] An "authentication token" is a temporary digital certificate that indicates that a user has been properly authenticated.

[2141] A "storage shelf" is a device that moves to a specified position to store or retrieve items.

[2142] "Destination information" is location information of the destination of the storage shelf designated by the user.

[2143] "Real-time data" is data that is updated immediately to indicate the current location and status of a storage shelf.

[2144] A "safety device" is a device that detects obstacles and stops the storage shelf safely while it is in operation.

[2145] An "error message" is a warning notification that is generated when a safety device detects an abnormality.

[2146] An "algorithm" is a series of calculation procedures that the server uses to calculate the optimal movement path of the storage shelf.

[2147] The "Dijkstra algorithm" is an algorithm based on graph theory that the server uses to calculate the shortest path through the storage shelves.

[2148] The system of the present invention is composed of a user, a terminal, and a server. The terminal operated by the user communicates with the server to control and configure the storage device, thereby enabling safe and efficient storage and retrieval of items.

[2149] First, the user starts the authentication process through the device application. The user enters their ID and password and presses the login button. The device encrypts the entered authentication information using the SSL / TLS protocol and sends it to the server. The server compares the received authentication information with its internal database and, if a match is found, generates an authentication token. This authentication token is sent to the device and stored on the device.

[2150] After authentication, the user can use the application interface to specify the destination of the storage shelf. The specified destination information is sent from the terminal to the server, and the server calculates the optimal movement route. This calculation uses the Dijkstra algorithm to calculate the shortest route that takes into account obstacles between the current storage shelf position and the specified destination. The server then sends the calculated route instructions to the storage device, and the storage device begins moving along the specified route.

[2151] During movement, real-time data from the storage device's sensors and encoders is sent to the server. The server organizes the received data and sends it to the terminal via the WebSocket protocol. The terminal displays the data in real time, showing the user the current location and status of the storage shelf. If necessary, the user can change the movement destination or instructions.

[2152] Furthermore, the operation of safety devices is checked periodically. The server checks data from the storage device's sensors at regular intervals to confirm the status of the safety devices. If an abnormality is detected, the server generates an error message such as "Obstacle detected" and immediately notifies the terminal. After receiving the notification, the user can take the necessary safety measures.

[2153] As a concrete example, if a user wants to move a storage shelf to "Position A," the device sends the information "Destination: Position A" to the server, and the server calculates the optimal route using the Dijkstra algorithm. Also, if an obstacle is detected while the storage shelf is moving, the sensor sends the data "Obstacle detected: Position X, Y" to the server, and an error message is generated.

[2154] Example prompt for a generative AI model:

[2155] "Please explain the SSL / TLS encryption used for login authentication. Please provide a detailed explanation including specific implementation examples."

[2156] The system allows users to efficiently and safely store and retrieve items in limited spaces, and its high load-bearing capacity and customizable modular structure make it suitable for a variety of needs.

[2157] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2158] Step 1: The user enters their ID and password.

[2159] The user opens the application on the terminal and enters their ID and password. The input is made into the text boxes on the login screen. The ID (e.g., "user123") and password (e.g., "password123") are recorded on the terminal as input data.

[2160] Step 2: The device encrypts the input and sends it to the server.

[2161] The terminal encrypts the ID and password entered by the user using the SSL / TLS protocol. The encrypted data is sent to the server using an HTTP POST request. At this time, the input data, ID and password, are converted into encrypted form.

[2162] Step 3: The server checks the database.

[2163] The server decrypts the received encrypted data and compares it with its internal database. Specifically, it searches the database for the corresponding user information and checks whether it matches the entered ID and password. If the comparison is successful, it sets a flag indicating "authentication successful."

[2164] Step 4: The server generates and sends an authentication token.

[2165] The server generates an authentication token based on the matching result. This authentication token is in JWT (JSON Web Token) format and contains the user's authentication information. The generated token is sent from the server to the terminal as an HTTP response, which notifies the user that authentication was successful.

[2166] Step 5: The user specifies the movement position.

[2167] The user uses the terminal application interface to specify the destination of the storage shelf. The "destination location information" (e.g., "Location A") is recorded as input data on the terminal. This information is treated as a movement command.

[2168] Step 6: The terminal sends the destination information to the server.

[2169] The terminal sends the specified destination information to the server using an HTTP POST request. The "destination information" as input data is sent to the server and serves as a trigger to start processing.

[2170] Step 7: The server calculates the optimal route.

[2171] The server calculates the optimal route based on the current storage shelf location and the specified destination. Specifically, it calculates the shortest route using the Dijkstra algorithm. The "current location" and "destination location" are processed as input data, and the "optimal route" is obtained as output.

[2172] Step 8: The server sends a transfer instruction to the storage device.

[2173] The server formats the calculated optimal route and sends a movement instruction to the storage device. The route information is sent to the storage device using an HTTP POST request, which causes the storage device to start moving.

[2174] Step 9: The server receives the real-time data.

[2175] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The "current location information" is sent to the server in real time as input data. This allows the current location of the storage shelf to be tracked.

[2176] Step 10: The server sends the data to the terminal.

[2177] The server organizes the received real-time data and sends it to the terminal using the WebSocket protocol. The "organized real-time data" is output to the terminal as input data, allowing the terminal to display the current status.

[2178] Step 11: The device will display the current status.

[2179] The terminal displays the received real-time data in a graphical user interface. The "real-time data" input is drawn on the screen, showing the user the current location and status of the storage shelves.

[2180] Step 12: The user changes the operation.

[2181] The user can check the location of the storage shelf during movement using the app and change the destination or instructions as necessary. The "new destination information" is recorded as input data on the device and then sent back to the server.

[2182] Step 13: The server checks the status.

[2183] The server periodically checks the data sent from the storage device's sensors to confirm the status of the safety devices. The "sensor information" is taken in as input data and a judgment is made as to whether the device is normal or abnormal.

[2184] Step 14: The server generates an error message.

[2185] If abnormal data is received from the sensor, the server generates an error message. The "abnormal information" is processed as input data, and an "error message" is created as output data. Specifically, a message such as "Obstacle detected" is generated.

[2186] Step 15: The server sends an error message to the terminal.

[2187] The server immediately sends the generated error message to the terminal. The "error message" is sent to the terminal as input data and notified in real time.

[2188] Step 16: The terminal notifies the user.

[2189] The terminal notifies the user of the error message by displaying a pop-up message, etc. The "error message" is displayed as input data, and the user is informed of the abnormality.

[2190] Step 17: The user takes safety measures.

[2191] After receiving the notification, the user can take the necessary safety measures, such as stopping the movement of the storage shelf in the app. This action allows the problem to be safely resolved.

[2192] (Application example 1)

[2193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2194] In logistics centers, to improve the efficiency of storing and retrieving goods, it is necessary to quickly and accurately control the position of storage shelves and storage equipment. However, conventional systems lack the ability to acquire real-time data and check the operation of safety devices, making it difficult to improve efficiency and safety. In addition, there is a need to ensure security and optimality in user authentication and route calculation.

[2195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2196] In this invention, the server includes terminal means for receiving user authentication information and transmitting it to the server, means for comparing the user authentication information received by the server with a database and generating an authentication token, means for transmitting the authentication token generated by the server to the terminal, means for transmitting storage device destination information specified by the user from the terminal to the server, means for the server to calculate an optimal movement route for the storage device and transmit a movement instruction to the storage device, means for transmitting real-time data of the storage device as it moves to the server, means for displaying the real-time data received from the server on the terminal, means for the server to periodically check the status of the safety device and generate an error message and notify the terminal if an abnormality is detected, means for using a JSON web token for login, and means for the server to calculate a movement route for the storage device, including a route calculation algorithm when transmitting instructions to the robot to the system, and notifying the user of an error message in real time as a safety measure. This enables efficient storage management through real-time data acquisition and highly safe operation.

[2197] "User authentication information" refers to information used by a user when accessing a system, and includes, for example, a user ID and password.

[2198] "Terminal means" refers to a device or apparatus that is operated by a user and that communicates with a server to send and receive data.

[2199] A "database" is a structured information collection system for managing and storing data used by an organization or system.

[2200] An "authentication token" is a code that indicates that a user has been authenticated, and is used for session management and access control.

[2201] "Storage equipment" refers to equipment or devices for storing and retrieving items.

[2202] "Destination information" is data relating to the location to which the storage equipment should be moved.

[2203] The "movement route" is the optimum route for the storage equipment to reach the designated destination.

[2204] "Real-time data" is up-to-date information about ongoing processes and conditions.

[2205] "Safety equipment" refers to devices and sensors used to ensure work safety.

[2206] An "error message" is a notification message that is generated when the system detects an abnormality or error.

[2207] "JSON Web Token" is a compact representation format used for user authentication and secure exchange of information.

[2208] "Storage equipment" means equipment or devices for the safe storage of items.

[2209] A "route calculation algorithm" is a method or calculation means for calculating the optimal route to a specified destination.

[2210] To implement this invention, a system including a server, a terminal, and a storage device is required. A specific embodiment of this system will be described below.

[2211] The server first manages the user authentication process. The user enters their ID and password on the device, which then encrypts them using the SSL / TLS protocol and sends them to the server. The server compares the received ID and password with its database, and if they match, generates a JWT (JSON Web Token) and sends it to the device. This achieves secure authentication.

[2212] Next, the destination information of the storage device specified by the user is sent from the terminal to the server. The server uses a route calculation algorithm to calculate the optimal route based on the current location of the storage device and the specified destination information. The server sends this route information to the storage device and issues a movement instruction.

[2213] When the storage device starts moving, real-time data from sensors and encoders is sent to the server. The server organizes this data and sends it to the terminal, which then displays the movement status to the user in real time. The user can change the movement destination or instructions as needed.

[2214] To enhance safety, the server periodically checks the status of safety devices. If an abnormality is detected, the server immediately generates an error message and notifies the terminal. The terminal receives this and notifies the user. The user receives the notification and can take safety measures.

[2215] For example, when a user requests on a terminal that "I want to move the shelf to position A3," the server calculates the optimal movement path and sends a movement command to the robot. If an obstacle is detected during movement, an error message is immediately generated and notified to the user.

[2216] Example prompt sentence:

[2217] "The user specified that they wanted the shelf to move to the A3 position."

[2218] This system enables efficient storage management and highly safe operation through real-time data acquisition.

[2219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2220] Step 1:

[2221] The user enters their ID and password on the device.

[2222] Input: User ID, Password

[2223] Output: Encrypted user ID and password

[2224] The terminal encrypts the ID and password entered by the user on the login screen using the SSL / TLS protocol and sends it to the server in a secure format.

[2225] Step 2:

[2226] The server checks the received user ID and password against the database.

[2227] Input: Encrypted user ID and password

[2228] Output: Authentication token

[2229] The server decodes the received user ID and password and checks them against the information stored in the database. If they match, the server generates a JWT (JSON Web Token).

[2230] Step 3:

[2231] The server sends the generated authentication token to the terminal.

[2232] Input: Authentication Token

[2233] Output: Authentication token sent to the device

[2234] The server sends the generated JWT to the device, which stores it and uses it for subsequent authentication.

[2235] Step 4:

[2236] The destination information of the storage device designated by the user is transmitted from the terminal to the server.

[2237] Input: Destination information (e.g., A3 position)

[2238] Output: Destination information sent to the server

[2239] The user uses the application interface to specify the destination of the storage device, and the terminal transmits this information to the server.

[2240] Step 5:

[2241] The server calculates the optimal movement route for the storage device and sends a movement instruction to the storage device.

[2242] Input: Destination information, current storage device location

[2243] Output: Optimal travel route, travel instructions

[2244] The server uses the current location and the specified destination information to execute a route calculation algorithm to calculate the optimal route, and then sends this movement instruction to the storage device.

[2245] Step 6:

[2246] Real-time data on the movement of stored equipment is sent to the server.

[2247] Input: Real-time data of storage equipment

[2248] Output: Real-time data sent to the server

[2249] As the storage equipment begins to move, real-time data from sensors and encoders is collected and sent to a server.

[2250] Step 7:

[2251] The real-time data received from the server is displayed on the terminal.

[2252] Input: Real-time data

[2253] Output: Real-time data displayed on the terminal

[2254] The server organizes the received real-time data and sends it to the terminal, which displays the data to the user in real time.

[2255] Step 8:

[2256] The server periodically checks the status of the safety equipment and, if there is an abnormality, generates an error message and notifies the terminal.

[2257] Input: Safety equipment status data

[2258] Output: Error message, user notification

[2259] The server periodically checks the status of the safety equipment, and if an abnormality is detected, it generates an error message that is immediately sent to the terminal, which then notifies the user.

[2260] Step 9:

[2261] The user receives a notification and takes safety measures.

[2262] Input: Error message

[2263] Output: User's response action

[2264] When the terminal notifies the user of an error message, the user receives it and can stop or change the operation as needed.

[2265] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2266] System configuration

[2267] The system of the present invention is composed of a user, a terminal, a server, and an emotion engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion engine recognizes the user's emotions and adjusts the system's response based on these.

[2268] Program processing overview and specific examples

[2269] User Authentication

[2270] 1. The user enters their ID and password on the device's login screen.

[2271] The user enters their ID and password on the app's login screen and presses the login button.

[2272] 2. The device encrypts the ID and password and sends them to the server.

[2273] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[2274] 3. The server checks the ID and password against the database.

[2275] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[2276] 4. The server generates an authentication token and sends it to the device.

[2277] The server sends the generated authentication token to the terminal, which stores it.

[2278] Determining the location of storage shelves

[2279] 1. The user specifies the destination of the storage shelf on the terminal.

[2280] Users select the location where they want to move the storage shelf using the app's interface.

[2281] 2. The terminal sends the specified destination information to the server.

[2282] The terminal transmits the destination information specified by the user to the server.

[2283] 3. The server calculates the optimal route based on the received information.

[2284] The server calculates the optimal route based on the current location of the storage shelf and the specified destination.

[2285] 4. The server sends a move instruction to the storage device.

[2286] The server sends a movement instruction to the storage device based on the calculated optimum route.

[2287] Shelf movement and control

[2288] 1. The server receives real-time data from the storage device.

[2289] When the storage device starts moving, it sends real-time data from sensors, encoders, etc. to a server.

[2290] 2. The server sends the received real-time data to the terminal.

[2291] The server organizes the real-time data received from the storage device and transmits it to the user's terminal.

[2292] 3. The terminal displays the real-time data to the user.

[2293] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves.

[2294] 4. The user changes the operation as needed.

[2295] Users can use the app to change the storage shelf's destination or stop it while it is in motion.

[2296] Checking the operation of safety devices

[2297] 1. The server checks the status of the safety device at regular intervals.

[2298] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule.

[2299] 2. The server generates an error message when an abnormality is detected.

[2300] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected."

[2301] 3. The server sends an error message to the terminal.

[2302] The server immediately sends any error messages it generates to the terminal.

[2303] 4. The terminal notifies the user of the error message.

[2304] The device will display the error message to the user using the app's notification function.

[2305] 5. The user receives a notification and takes corrective action.

[2306] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[2307] Implementing the Emotion Engine

[2308] 1. The device recognizes the user's emotions.

[2309] The device's camera, microphone, and other sensors are used to recognize emotions from the user's facial expressions, tone of voice, etc.

[2310] 2. The device sends the emotion data to the emotion engine.

[2311] The device sends the collected emotion data to the emotion engine.

[2312] 3. The emotion engine analyzes the emotion data.

[2313] The emotion engine analyzes the received data to determine the user's current emotional state.

[2314] 4. The server determines the necessary system response based on the emotion engine's analysis results.

[2315] Based on the data from the emotion engine, the server determines an appropriate response, such as simplifying the operating procedure if the user is feeling stressed.

[2316] 5. The server sends the response to the device.

[2317] The server sends an appropriate response to the terminal, which displays the response to the user.

[2318] 6. The user takes action based on the device's response.

[2319] The user can view the response from the terminal and take appropriate action based on it.

[2320] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[2321] The processing flow will be explained below.

[2322] Program processing steps

[2323] User Authentication

[2324] Step 1:

[2325] The user enters their ID and password on the login screen of the device.

[2326] The user enters their ID and password on the app's login screen and presses the login button.

[2327] Step 2:

[2328] The device encrypts the ID and password and sends them to the server.

[2329] The terminal encrypts the entered ID and password using SSL / TLS and sends it to the server.

[2330] Step 3:

[2331] The server checks the ID and password against the database.

[2332] The server searches the database for the relevant user information and, if there is a match, generates an authentication token.

[2333] Step 4:

[2334] The server generates an authentication token and sends it to the device.

[2335] The server sends the generated authentication token to the terminal, which stores it.

[2336] Determining the location of storage shelves

[2337] Step 1:

[2338] The user specifies the destination of the storage shelf on the terminal.

[2339] The user selects a specific location within the app's interface where they want to move the storage shelf, such as "to the left of the living room."

[2340] Step 2:

[2341] The terminal transmits the designated destination information to the server.

[2342] The device sends destination information specified by the user to the server, including the coordinates and direction of the destination.

[2343] Step 3:

[2344] The server calculates the optimal travel route based on the information received.

[2345] The server calculates the optimal route based on the current location of the storage shelf and the specified destination, using the A algorithm or Dijkstra algorithm.

[2346] Step 4:

[2347] The server sends a movement instruction to the storage device.

[2348] The server then sends movement instructions to the storage device based on the calculated optimal route, including start and stop times for the movement, and alternative routes.

[2349] Shelf movement and control

[2350] Step 1:

[2351] A server receives real-time data from the storage device.

[2352] When the storage device starts moving, it sends real-time data from sensors and encoders to the server, including its current position, moving speed, and encoder counts.

[2353] Step 2:

[2354] The server transmits the received real-time data to the terminal.

[2355] The server organizes real-time data received from the storage device and sends it to the user's device, including the device's movement trajectory and current location information.

[2356] Step 3:

[2357] The terminal displays the real-time data to the user.

[2358] The device displays the data received from the server on the app, visualizing the current location and status of the storage shelves to the user. For example, the storage shelf icon can be moved in real time on the app.

[2359] Step 4:

[2360] The user changes the operation as needed.

[2361] Users can use the app to change the storage shelf's destination or stop it while it is moving, for example, by issuing commands such as "Stop moving" or "Move to another location."

[2362] Checking the operation of safety devices

[2363] Step 1:

[2364] The server checks the status of the safety device at regular intervals.

[2365] The server periodically checks the status of the storage device's safety sensors and other safety devices based on a schedule, with check intervals set to about 10 seconds.

[2366] Step 2:

[2367] The server generates an error message when an abnormality is detected.

[2368] If the server detects an abnormality from the sensor information, it generates an error message such as "Obstacle detected," along with the type and location of the abnormality.

[2369] Step 3:

[2370] The server sends an error message to the terminal.

[2371] The server immediately sends any error messages it generates to the terminal.

[2372] Step 4:

[2373] The terminal notifies the user of the error message.

[2374] The device will display the error message to the user using the app's notification feature, which may include sound and / or vibration.

[2375] Step 5:

[2376] The user receives a notification and takes safety measures.

[2377] The user receives the error message, stops the operation of the storage device as necessary, and performs safety checks.

[2378] Implementing the Emotion Engine

[2379] Step 1:

[2380] The device recognizes the user's emotions.

[2381] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice, and sends them to the emotion engine. Evaluation items include smiles, tone of voice, and facial muscle movements.

[2382] Step 2:

[2383] The device transmits the emotion data to the emotion engine.

[2384] The device transmits the collected emotion data to the emotion engine in real time.

[2385] Step 3:

[2386] The emotion engine analyzes the emotion data.

[2387] The emotion engine analyzes the received data with a proprietary algorithm to identify the user's current emotional state, specifically using a machine learning model to classify emotional states such as "stressed" or "relaxed."

[2388] Step 4:

[2389] The server determines the necessary system response based on the analysis results of the emotion engine.

[2390] Based on the results of the emotion engine, the server determines an appropriate response, such as simplifying the operation procedure if the user is feeling stressed.

[2391] Step 5:

[2392] The server sends a response to the terminal.

[2393] The server sends an appropriate response to the terminal, which displays the response to the user.

[2394] Step 6:

[2395] The user takes action according to the terminal's response.

[2396] The user can view the response from the device and take appropriate action based on it, for example by displaying a suggestion such as "Start a relaxing massage."

[2397] This allows the user to receive appropriate support according to their emotional state, and allows them to operate the movable storage device safely and efficiently.

[2398] Example 2

[2399] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2400] Current mobile storage systems have basic functions such as user authentication, moving storage shelves, displaying real-time data, and monitoring safety devices, but they lack system responses that take user emotions into consideration, and there is a need for improved user experience. Another issue is the lack of a mechanism for quickly notifying users when an abnormality is detected. This can lead to complex operations and insufficient safety.

[2401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2402] In this invention, the server includes means for comparing user authentication information with a database and generating an authentication token, means for calculating an optimal movement route for the storage shelf, means for the server to periodically check the status of the safety device and, if an abnormality is found, generate an error message and notify the terminal, means for the terminal to recognize the user's emotion and send the emotion data to an analysis engine, and means for the analysis engine to analyze the emotion data, the server to determine a system response, and display the response on the terminal. This provides an appropriate system response that takes the user's emotion into consideration, making it possible to simplify operation and improve safety.

[2403] "User authentication information" refers to information such as ID and password required for a user to log in to a system.

[2404] A "terminal" is a device operated by a user, and is a device that communicates with a server to send and receive various data.

[2405] A "server" is a central computer system that connects to a database, performs various calculations and data processing, and sends instructions and information to terminals and other devices.

[2406] An "authentication token" is electronic information that is generated after the server has confirmed the authentication of a user, and is used to maintain the authentication status of the user.

[2407] A "storage shelf" is a movable storage device, which is an item storage device that moves to a specified position based on a user's instructions.

[2408] "Destination information" is information relating to the new location of the storage shelf designated by the user.

[2409] The "optimal movement route" is the most efficient route from the current storage shelf position to the movement destination specified by the user.

[2410] "Real-time data" refers to data that is updated immediately, such as location information and sensor information generated when a storage device moves.

[2411] "Safety devices" are sensors and mechanical protection devices that ensure safety during operation of the storage device.

[2412] An "error message" is a warning or error message that is generated when the system detects an abnormality and is notified to the user.

[2413] "Emotion data" refers to data related to the user's emotions, such as facial expressions and tone of voice, acquired through the device's sensors and camera.

[2414] An "analysis engine" is software or hardware for analyzing emotion data to identify the user's emotional state.

[2415] "System response" refers to the system's actions and display messages that the server determines based on the user's operations and emotional state.

[2416] The system of the present invention is composed of a user, a terminal, a server, and an emotion analysis engine. The terminal operated by the user communicates with the server to control and configure the storage device. Furthermore, the emotion analysis engine recognizes the user's emotions and adjusts the system's response based on these.

[2417] Hardware and Software Configuration

[2418] Specific examples of system hardware include smartphones and PCs as terminals. These terminals are equipped with cameras, microphones, touchscreens, etc., and provide a means of interface with users. The server uses a high-performance database management system (e.g., MySQL, PostgreSQL), and cloud services can also be used.

[2419] The system's software implementation uses the following technologies:

[2420] Use SSL / TLS encryption (e.g., OpenSSL library) for user authentication.

[2421] RESTful API and WebSocket are used to send and receive data.

[2422] The optimal movement path of the storage shelf is calculated using Dijkstra's algorithm or A algorithm (e.g., NetworkX library).

[2423] Use a JavaScript library (e.g. D3.js) to display real-time data.

[2424] The Python Scikit-learn library is used to check the operation of various safety devices and detect abnormalities.

[2425] OpenCV and Google Cloud Speech API are used to recognize user emotions, and TensorFlow and PyTorch are used to analyze emotion data.

[2426] Example of a system

[2427] Here is a specific example of how the system works. When a user wants to move a storage shelf from the "living room" to the "office," they perform the following operations:

[2428] 1. The user selects the "Office" area on the smartphone app interface and taps the Go button.

[2429] 2. The device sends the selected destination information to the server as an HTTP POST request.

[2430] 3. The server uses the NetworkX library to calculate the optimal route using Dijkstra's algorithm and sends movement instructions to the storage device using the MQTT protocol.

[2431] 4. When the storage device starts moving, it sends real-time data (location information and sensor information) to the server via the MQTT protocol.

[2432] 5. The server organizes the received real-time data and sends it to the user's device using WebSocket.

[2433] 6. The device uses D3.js to visualize the received data on the app screen, showing the user the current location and status of the storage shelf.

[2434] 7. If the user feels tired or stressed while traveling, the device's camera and microphone will collect emotional data and send it to the emotion analysis engine.

[2435] 8. The emotion analysis engine uses TensorFlow to analyze the emotion data and notify the server that the user is feeling stressed.

[2436] 9. Based on the emotion analysis results, the server generates a guide message that simplifies the operation procedure and sends it to the device via WebSocket.

[2437] 10. The user can operate the storage device efficiently by following the guide messages from the terminal and performing the necessary operations.

[2438] Prompt Sentence Examples

[2439] Possible prompts for a generative AI model include:

[2440] "How can I design an application that automatically adjusts its system response based on user emotion?"

[2441] "Please give us a concrete example of an algorithm that uses an emotion analysis engine to detect a user's stress level and simplify the operation procedure."

[2442] This allows the user to safely and efficiently operate the movable storage device while receiving appropriate support according to their emotional state.

[2443] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2444] Step 1:

[2445] The user enters their ID and password on the login screen of the device.

[2446] Input: ID and password entered by the user

[2447] How it works: The user enters their ID and password on the login screen of their smartphone or computer and clicks the login button.

[2448] Output: Device encrypted credentials

[2449] Step 2:

[2450] The device encrypts the entered ID and password and sends them to the server.

[2451] Input: ID and password entered by the user

[2452] How it works: The device encrypts the ID and password using the SSL / TLS protocol (e.g., using the OpenSSL library) and sends it to the server as an HTTP POST request.

[2453] Output: Encrypted credentials sent to the server

[2454] Step 3:

[2455] The server checks the ID and password against the database.

[2456] Input: Encrypted credentials

[2457] What it does: The server accesses a database (e.g. MySQL or PostgreSQL) and runs a SQL query to verify that the entered ID and password match.

[2458] Output: Authentication result (success or failure)

[2459] Step 4:

[2460] The server generates an authentication token and sends it to the device.

[2461] Input: If authentication result is successful

[2462] How it works: The server generates an authentication token using a JWT (JSON Web Token) library and sends it to the device as an HTTP response. The device stores this token in local storage.

[2463] Output: Authentication token

[2464] Step 5:

[2465] The user specifies the destination of the storage shelf on the terminal.

[2466] Input: User-selected destination information

[2467] How it works: The user selects the new location for the storage shelf in the in-app interface and clicks a button to confirm the move.

[2468] Output: Destination information

[2469] Step 6:

[2470] The terminal transmits the designated destination information to the server.

[2471] Input: Destination information

[2472] Operation: The device sends the specified destination information in JSON format to the server. This is done using a RESTful API as an HTTP POST request.

[2473] Output: Destination information is sent to the server

[2474] Step 7:

[2475] The server calculates the optimal travel route based on the information received.

[2476] Input: Current location and specified destination information

[2477] How it works: The server uses Dijkstra's algorithm or A algorithm (using Python's NetworkX library, for example) to calculate the optimal route from the current location to the specified destination.

[2478] Output: Optimal route information

[2479] Step 8:

[2480] The server sends a movement instruction to the storage device.

[2481] Input: Optimal route information

[2482] Operation: The server sends the calculated optimal route information to the storage device. This is done using the MQTT protocol.

[2483] Output: Movement instructions are sent to the storage device

[2484] Step 9:

[2485] Real-time data is sent to the server as the storage device moves.

[2486] Input: Movement instructions

[2487] Operation: The storage device sends position and status information obtained from motors and sensors to a server. MQTT is used as the communication protocol.

[2488] Output: Real-time data

[2489] Step 10:

[2490] The real-time data received by the server is displayed on the terminal.

[2491] Input: Real-time data

[2492] Operation: The server organizes the received data and sends it to the device in real time using WebSocket.

[2493] Output: Organized real-time data

[2494] Step 11:

[2495] The terminal displays the real-time data to the user.

[2496] Input: Organized real-time data

[2497] Operation: The device uses the JavaScript D3.js library to display the received real-time data as graphs and maps on the app screen.

[2498] Output: Visualized current shelf location and status

[2499] Step 12:

[2500] The user changes the operation as needed.

[2501] Input: Real-time data and current operating status

[2502] How it works: The user can change the position and speed of the storage shelf through the app interface. The changed instructions are immediately sent to the server.

[2503] Output: Modified operating instructions

[2504] Step 13:

[2505] The server periodically checks the status of the safety device.

[2506] Input: Safety device sensor information

[2507] How it works: The server periodically schedules tasks to obtain information from safety sensors, for example, using Crontab or a system daemon.

[2508] Output: Safety device status data

[2509] Step 14:

[2510] The server generates an error message when an abnormality is detected.

[2511] Input: Safety device status data

[2512] How it works: The server uses Python's Scikit-learn library to perform anomaly detection and generates an error message if an anomaly is detected.

[2513] Output: Error message

[2514] Step 15:

[2515] The server sends an error message to the terminal.

[2516] Input: Error message

[2517] How it works: The server uses a RESTful API to send an error message to the device.

[2518] Output: Error messages are sent to ...

Claims

1. a terminal means for receiving user authentication information and transmitting it to a server; a means for the server to compare the received user authentication information with a database and generate an authentication token; A means for transmitting the authentication token generated by the server to the terminal; means for transmitting, from the terminal to the server, information on the storage shelf destination designated by the user; A means for the server to calculate an optimum movement route for the storage shelf and transmit a movement instruction to the storage device; means for transmitting real-time data when the storage device is moving to a server; means for displaying on a terminal the real-time data received from the server; A means for the server to periodically check the status of the safety device and, if there is an abnormality, generate an error message and notify the terminal; A system including:

2. 10. The system of claim 1, wherein the storage shelf moves horizontally and vertically, is constructed using lightweight materials, and has a plurality of rollers and guide rails.

3. 2. The system of claim 1, wherein the server includes an algorithm for calculating a travel path and notifies the user when an abnormality is detected.

Images