system

Abstract

Provide a system. [Solution] a means for collecting data from sensors installed in the living space; an artificial intelligence means for analyzing data collected from the sensors and proposing a storage method; A means for a user to input the storage location of an item; a database means for recording and managing the input storage location and item information; means for searching for and notifying the location of an item based on the database information; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to a method and system for efficiently using limited living space and optimizing item storage and inventory management in modern single-person and dual-income households. Specifically, the purpose is to solve problems such as forgetting where items are, ineffective use of storage space, and being too busy to find time to organize things. The purpose of the present invention is to solve these problems and enable users to live more comfortable and efficient lives. [Means for solving the problem]

[0005] The present invention provides the following means: a system including means for collecting data from sensors installed in a living space, artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods, means for users to input storage locations for items, database means for recording and managing the input storage locations and item information, and means for searching for and notifying the location of items based on the database information. This system enables efficient use of space and optimized item management, significantly reducing the time and effort required for users to store items. In addition, the artificial intelligence means learns the user's lifestyle patterns and preferences through deep learning and constantly optimizes its suggestions. The sensors include camera sensors and RFID sensors, enabling accurate location data acquisition and detailed item identification.

[0006] A "sensor" is a device used to detect physical conditions within a living space and collect data, including camera sensors and RFID sensors.

[0007] A "data collection means" is a method or device for acquiring data from a living space using sensors.

[0008] "Artificial intelligence means" refers to artificial intelligence algorithms or programs used to analyze collected data and suggest efficient storage methods.

[0009] "User input means" refers to an interface or device that allows a user to input the storage location of an item. For example, this refers to a smartphone app or tablet application.

[0010] "Database means" refers to a database system for recording and managing storage location and item information entered by users.

[0011] "Search and notification means" refers to a method or device for searching for the location of an item based on database information and notifying the user of that information.

[0012] "Storage Suggestions" means recommendations generated by artificial intelligence means regarding where and how best to store items.

[0013] "Deep learning" is a type of AI technology used to learn users' lifestyle patterns and preferences and improve algorithms.

[0014] "Natural language processing" is a technology for processing and understanding user input in natural language, and is used for searching for items and inventory management.

[0015] "Space utilization optimization" refers to optimizing storage methods and layouts to efficiently utilize limited living space. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to propose efficient storage methods and support the management and retrieval of items. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0038] Overall system configuration

[0039] Sensor installation and data collection

[0040] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0041] Data analysis and storage proposals

[0042] The server preprocesses the collected sensor data and passes it as input parameters to an AI algorithm, which then analyzes the data and generates the optimal storage method. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0043] Suggestion notifications and interface

[0044] The device notifies the user of storage suggestions generated by the AI ​​algorithm. The suggestions are displayed as pop-up notifications or highlighted on the UI via the smartphone or tablet app. The user then uses these suggestions to store items in the designated locations.

[0045] Recording and managing product information

[0046] When a user puts an item away, they input the storage location into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0047] Search and Notifications

[0048] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0049] Specific system operation examples

[0050] Example 1: Organizing your living room

[0051] 1. Sensor installation: Install a camera sensor and an RFID sensor in the living room.

[0052] 2. Data collection: The server collects furniture layout and item location data from sensors.

[0053] 3. Data analysis: The server uses an AI algorithm to analyze, for example, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0054] 4. Proposal notification: The terminal notifies the user of this proposal.

[0055] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0056] 6. Recording information: The server records this information in a database.

[0057] Example 2: Product search

[0058] 1. Search request: The user asks the app, "Where is the remote?"

[0059] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0060] 3. Notification: The device notifies the user of this information.

[0061] This system allows for efficient storage method suggestions, item management, and item searching, improving the quality of life for users.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server.

[0065] Step 2:

[0066] The server preprocesses the collected data: in the case of image data, it uses object recognition algorithms to detect items and extract location information, and in the case of RFID data, it determines the item's ID and location.

[0067] Step 3:

[0068] The server inputs the pre-processed data into an artificial intelligence algorithm to analyze the living space. The analysis results in an efficient storage solution. Specifically, it calculates where and how to store items to maximize space utilization.

[0069] Step 4:

[0070] The device notifies the user of the storage suggestions received from the server, and the suggestions are presented to the user via a smartphone or tablet app as pop-up notifications or highlighted on the interface.

[0071] Step 5:

[0072] The user stores the items in the designated locations according to the suggested storage method, for example, by following instructions such as "store the books in the bookshelf in the living room."

[0073] Step 6:

[0074] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0075] Step 7:

[0076] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0077] Step 8:

[0078] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0079] Step 9:

[0080] The server searches the database and obtains the location information of the relevant item. For example, the server obtains location information such as "The remote control is in the drawer of the table in the living room."

[0081] Step 10:

[0082] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0083] Step 11:

[0084] The server collects user feedback and updates the AI ​​algorithm, allowing it to learn and improve the accuracy of future suggestions.

[0085] Example 1

[0086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0087] Efficient storage and retrieval of items is a crucial issue in modern living spaces, but it is difficult to manage a variety of items easily while making effective use of limited space. Conventional systems require users to manage items manually, which takes time and effort, and users often forget where they have stored them. Furthermore, there is a lack of efficient storage suggestions, making it difficult to make optimal use of living space.

[0088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0089] In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors, artificial intelligence means for analyzing the preprocessed data and proposing storage methods, means for notifying the user of the proposed storage methods, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, and means for searching for and notifying the location of items based on the database information, thereby enabling efficient storage method suggestions, item management, and rapid search.

[0090] "Sensors" are devices used to collect location information about objects and furniture in living spaces, including camera sensors and RFID sensors.

[0091] "Preprocessing" is the process of shaping or filtering data collected from sensors to make it easier to analyze.

[0092] "Artificial intelligence means" refers to technology for generating and proposing efficient storage methods based on preprocessed data, and machine learning algorithms are included in this section.

[0093] The "notification means" is a method for informing the user of the storage method suggestions generated by the server, and is mainly an application on a smartphone or tablet.

[0094] "Database means" means a system for recording and managing the location information of items and other related information entered by users.

[0095] A "search means" is a system that has the ability to identify the location of an item based on information in a database and notify the user of that information.

[0096] The "Smart Organizing Support AI System" of this invention is a system that uses sensors and artificial intelligence installed in living spaces to suggest efficient storage methods and assist with managing and retrieving items. The system aims to provide optimal storage methods for making effective use of limited space and reduce the effort required for users to search for items.

[0097] Overall system configuration

[0098] Sensor installation and data collection

[0099] The server collects real-time data from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0100] Data Preprocessing

[0101] The server pre-processes the collected raw data: image data from the camera sensors is analyzed using image processing algorithms to identify furniture layout and object locations, and RFID data is decoded to extract item identification information.

[0102] Organizing and storage proposal generation

[0103] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0104] proposal notification

[0105] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The user uses this information to store the items in the specified location.

[0106] Recording the location of items

[0107] When a user places an item in a designated location, they input the location information into the app. For example, they might input "Put new books in the bookshelf in the living room." This information is then sent back to the server and recorded in the database.

[0108] Item location search and notification

[0109] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map.

[0110] Specific system operation examples

[0111] Living room organization

[0112] 1. A user installs a camera sensor on the ceiling of their living room and places RFID sensors on key furniture pieces.

[0113] 2. The server periodically collects image data from the camera sensor and item ID data from the RFID sensor.

[0114] 3. The server uses image analysis algorithms to analyze the furniture layout and extract item identification information from the RFID data.

[0115] 4. The server uses an AI algorithm to analyze the situation and generate a suggestion: "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0116] 5. The device will then display a pop-up notification suggesting to the user that "it would be better to store magazines in the storage box next to the sofa."

[0117] 6. The user enters "Put the magazine in the box next to the sofa" into the app, and the item's location information is sent to the server.

[0118] 7. The server records the transmitted information in a database.

[0119] Product Search

[0120] 1. The user types into the app, "Where's the remote?"

[0121] 2. The server searches the database and obtains the location information: "The remote control is in the table drawer in the living room."

[0122] 3. Your device will display a pop-up notification with this information, possibly showing your location along with a floor plan of your living room.

[0123] Example prompts for generative AI models

[0124] "Design an AI algorithm that suggests efficient ways to store items."

[0125] "Please explain the program flow for building an inventory management system using data collected from sensors."

[0126] This system allows users to organize their living space and manage their belongings simply and efficiently, thereby improving the quality of their lives.

[0127] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0128] Step 1: Sensor installation and data collection

[0129] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. The input is raw data from the camera sensors and RFID sensors, and the output is the captured image data and ID information.

[0130] Step 2: Preprocessing the data

[0131] The server preprocesses the collected raw data. Specifically, it analyzes the image data using image processing algorithms to identify furniture layout and object locations. It also decodes the RFID data to extract item identification information. The inputs are raw data from camera sensors and RFID sensors, and the output is preprocessed location and identification information.

[0132] Step 3: Generate organization and storage proposals

[0133] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room. The inputs are preprocessed location information and identification information, and the output is a specific storage suggestion.

[0134] Step 4: Proposal Notification

[0135] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The input is the storage suggestions from the server, and the notified suggestion information is obtained as output.

[0136] Step 5: Record the location of the item

[0137] When a user stores an item in a designated location, they input the location information into the app. For example, they might enter "Store new books in the living room bookshelf." This information is sent back to the server and recorded in the database. The input is the location information from the user, and the output is the information recorded in the database.

[0138] Step 6: Search for and notify location of item

[0139] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map. The input is the user's question, and the output is the location information of the search results.

[0140] By clarifying what data processing and calculations are performed based on the input data at each step and what output is ultimately obtained, it becomes easier to understand the flow of the entire system.

[0141] (Application example 1)

[0142] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Logistics centers handle a wide variety of products in large quantities, so they need to propose optimal storage methods and efficiently search for them. With conventional systems, product placement within the warehouse was inefficient, and locating products took a lot of time and effort. This resulted in reduced operational efficiency and a serious labor shortage.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0145] In this invention, the server includes means for collecting data from sensors installed in the living space, artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods, means for users to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching and notifying the location of items based on the database information, and means for supporting the storage and search of packages at the logistics center, thereby enabling efficient management of packages and rapid search at the logistics center.

[0146] A "sensor" is a device for collecting data from a physical space, and includes, for example, a camera sensor or an RFID sensor.

[0147] "Means for collecting data" refers to a system that has the function of collecting data obtained from sensors and sending it to a server.

[0148] "Artificial intelligence means" refers to algorithms or software that analyze collected data and make recommendations or decisions according to specific purposes.

[0149] "Means for users to input the storage location of items" refers to an interface where users provide information about storage locations and items, which has the function of transmitting the information to a server.

[0150] "Database means" refers to a storage device and management system for recording and managing input storage location and item information.

[0151] "Means for searching for and notifying the location of an item" refers to a system that has the function of searching for the location of an item based on information recorded in a database and notifying the user of the results.

[0152] "Means to support the storage and retrieval of goods in a logistics center" refers to functions and systems that assist in the optimal placement and rapid retrieval of goods within a logistics center.

[0153] The "Smart Organizing Support AI System" based on this invention supports efficient storage and retrieval of packages in logistics centers. This system analyzes data collected from sensors and proposes optimal storage methods to improve the efficiency of package management and retrieval. A specific form of the system that realizes this invention will be described.

[0154] Sensor installation and data collection

[0155] The sensors installed in the distribution center include camera sensors and RFID sensors. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to a server via the Internet.

[0156] Data analysis and storage proposals

[0157] The server preprocesses the collected sensor data and stores it in a database. The preprocessed data is input into a generative AI model, where an algorithm is applied to suggest optimal storage locations. This generates specific suggestions, such as "Place the new product on shelf B-2." This analysis uses a machine learning algorithm using TENSORFLOW (registered trademark).

[0158] Suggestion notifications and interface

[0159] The server then sends the generated storage suggestions to smartphones and tablets. Warehouse staff receive the suggestions via pop-up notifications or highlighted displays on the UI, and store the items in the designated locations accordingly.

[0160] Recording and managing product information

[0161] After storing an item in a designated location, the user enters the information into the terminal. For example, they might enter "Store item X on shelf B-2." This information is sent to the server and recorded in a database. This allows for centralized management of item information within the logistics center and allows for real-time updates.

[0162] Search and Notifications

[0163] When a user wants to find a specific item, they enter a search query into their device. For example, they ask, "Where is item X?" The server searches the database and retrieves the location of the item. The search results are sent to the device and displayed to the user. In some cases, a visual map display is also displayed.

[0164] Specific examples

[0165] 1. Data Collection

[0166] Camera sensors are installed inside the warehouse to capture images of shelf layout and inventory status.

[0167] RFID sensors are used to scan the RFID tags on each product to collect information.

[0168] 2. Data Analysis

[0169] For example, the AI ​​will make suggestions such as, "This shelf has a weight limit, so move heavy items to another shelf."

[0170] 3. Proposal Notice

[0171] You receive a pop-up notification on your smartphone app saying, "Please place the new product on shelf B-2."

[0172] 4. Information Records

[0173] The warehouse staff enters "Place product X on shelf B-2" into the app and records it on the server.

[0174] 5. Product Search

[0175] Type into the app, "Where is product X?"

[0176] The server notifies, "Product X is on shelf B-2."

[0177] Prompt Sentence Examples

[0178] "Generate helpful storage suggestions for users in your distribution center. This should take into account the weight, size, and current inventory of an item and suggest the best storage location. For example, suggest placing heavy items on lower shelves and lighter items on higher shelves."

[0179] This enables the system to efficiently manage and quickly search for packages within the logistics center.

[0180] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0181] Step 1: Data collection

[0182] The server collects data from camera sensors and RFID sensors installed in the distribution center. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to the server via the Internet. The input is real-time data from the sensors, and the output is data sent to the server.

[0183] Step 2: Data Preprocessing

[0184] The server preprocesses the collected image data and RFID tag data. This process involves noise removal, data format conversion, and extraction of necessary data. The input is the collected sensor data, and the output is the preprocessed data.

[0185] Step 3: Data analysis

[0186] The server inputs the preprocessed data into a generative AI model and performs analysis to suggest optimal storage locations. For example, TensorFlow is used to consider the weight and size of items and current inventory status to suggest the optimal shelf. The input is the preprocessed data, and the output is storage suggestions generated by the AI.

[0187] Step 4: Notification of storage proposal

[0188] The server notifies the generated storage suggestions to the smartphone or tablet device, which then communicates the specific suggestions to the user through pop-up notifications or highlighting on the UI. The input is the suggestions generated by the AI, and the output is the notified suggestions.

[0189] Step 5: Recording product information

[0190] The user stores the items in the designated locations according to the suggestions and enters the information into the terminal. The terminal sends this information to the server, which records it in the database. The input is the item information sent by the user, and the output is the information recorded in the database.

[0191] Step 6: Search for items

[0192] When a user wants to find a specific item, they input a search query into their device. The server searches the database and obtains the location information of the corresponding item. The obtained information is sent to the device and notified to the user. The input is the user's search query, and the output is the location information of the item.

[0193] These processing steps result in efficient package management and rapid retrieval at the logistics center.

[0194] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0195] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with the management and retrieval of items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. This system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0196] Overall system configuration

[0197] Sensor installation and data collection

[0198] The server collects data in real time from camera and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server. The emotion engine also uses sensors to detect the user's tone of voice, facial expressions, and gestures.

[0199] Data analysis and storage proposals

[0200] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm. The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts its suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0201] Suggestion notifications and interface

[0202] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[0203] Recording and managing product information

[0204] When a user puts an item away, they input the storage location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0205] Search and Notifications

[0206] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0207] Specific system operation examples

[0208] Example 1: Organizing your living room

[0209] 1. Sensor installation: Install a camera sensor, RFID sensor, and emotion recognition sensor in the living room.

[0210] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0211] 3. Data analysis: The server generates storage suggestions using AI algorithms, and an emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0212] 4. Proposal notification: The terminal notifies the user of this proposal.

[0213] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0214] 6. Recording information: The server records this information in a database.

[0215] Example 2: Product search

[0216] 1. Search request: The user asks the app, "Where is the remote?"

[0217] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0218] 3. Notification: The device notifies the user of this information.

[0219] This system allows for efficient storage method suggestions, item management, and search, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0220] The processing flow will be explained below.

[0221] Step 1:

[0222] The server collects data in real time from camera sensors, RFID sensors, and emotion recognition sensors installed in the living space. The camera sensors capture image data of the room and furniture layout, while the RFID sensors collect ID data of items. The emotion recognition sensors analyze the user's tone of voice, facial expressions, and gestures to obtain emotional data.

[0223] Step 2:

[0224] The server preprocesses the collected data: for image data, it uses object recognition algorithms to detect items and extract location information; for RFID data, it identifies item IDs and locations; and for emotional data, it analyzes emotional states from voice, facial expressions, and gestures.

[0225] Step 3:

[0226] The server inputs the preprocessed data into an artificial intelligence algorithm to analyze the space usage. As a result of the analysis, it generates an efficient storage method. For example, it makes a specific suggestion such as "It would be good to store textbooks on the shelf next to the sofa in the living room." At this time, an emotion engine adjusts the suggestion based on the user's emotional state. For example, if the user is feeling stressed, it suggests a storage method that is simple and easy to access.

[0227] Step 4:

[0228] The device then notifies the user of the storage suggestions received from the server. The suggestions are displayed as notifications or highlighted on the interface via the smartphone or tablet app. For example, the device may notify the user that "it would be convenient to store textbooks on the shelf on the right side of the living room."

[0229] Step 5:

[0230] The user stores items in the designated location according to the storage method suggested by the user. For example, "store books in the bookshelf in the living room."

[0231] Step 6:

[0232] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0233] Step 7:

[0234] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0235] Step 8:

[0236] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0237] Step 9:

[0238] The server searches the database and obtains the location information of the relevant item. For example, the location information is obtained in the form of "The remote control is in the drawer of the table in the living room."

[0239] Step 10:

[0240] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0241] Step 11:

[0242] The server collects user feedback and updates the AI ​​algorithm and emotion engine, allowing it to learn and improve the accuracy of future suggestions.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] Currently, there is a lack of efficient methods for storing and managing items in living spaces. Furthermore, it is often time-consuming for users to search for items, making it difficult to effectively utilize living space. Furthermore, existing systems do not provide user-friendly experiences because they do not take into account the user's emotional state. A new system is needed to solve these problems.

[0246] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors and passing it to an artificial intelligence algorithm, means for notifying the user of the storage method generated by the artificial intelligence algorithm, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching for and notifying the location of items based on the database information, and means for analyzing emotional data and making suggestions according to the user's emotional state. This enables efficient storage method suggestions, item management and search, and user-friendly suggestions according to the user's emotional state.

[0247] "Living space" refers to the space and environment in which people live their daily lives.

[0248] A "sensor" refers to a device that detects a physical phenomenon and outputs that information as an electrical signal.

[0249] "Data preprocessing" refers to processes such as trimming, filtering, and decoding of raw data collected from sensors to make it easier to analyze.

[0250] "Artificial intelligence algorithms" refers to mathematical and statistical methods for analyzing data to perform specific tasks, and specifically includes machine learning and deep learning techniques.

[0251] "Storage method" refers to the techniques and procedures for arranging items efficiently and optimally.

[0252] "Notification" refers to the act of notifying a specific device or user of certain information.

[0253] "Input means" refers to the method or device by which a user provides information to a system.

[0254] A "database" refers to a system that systematically records and manages information and enables it to be searched and retrieved as needed.

[0255] "Emotional data" refers to information that indicates a user's emotional state, such as tone of voice, facial expressions, and gestures.

[0256] "User-friendly" refers to a design concept that allows users to operate intuitively and comfortably.

[0257] The "Smart Organizing Support AI System" of this invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with managing and searching for items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items.

[0258] Overall system configuration

[0259] Sensor installation and data collection

[0260] The server collects data in real time from camera sensors (e.g., Nest Cam) and RFID sensors (e.g., Impinj Speedway R420) installed in the living space. Image data captured by the camera sensors and item ID data acquired by RFID sensors are sent to the server. In addition, sensors used for the emotion engine (e.g., Intrinsic RealSense D435) detect the user's tone of voice, facial expressions, and gestures.

[0261] Data analysis and storage proposals

[0262] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts the suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0263] Suggestion notifications and interface

[0264] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app (e.g., iOS app or Android app) as pop-up notifications or highlighted on the UI. The user uses these suggestions to store items in the designated locations.

[0265] Recording and managing product information

[0266] When a user puts an item away, they input the storage location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in a database (e.g., MySQL (registered trademark) or PostgreSQL).

[0267] Search and Notifications

[0268] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0269] Specific system operation examples

[0270] Example 1: Organizing your living room

[0271] 1. Sensor installation: Install a camera sensor (e.g., Nest Cam), an RFID sensor (e.g., Impinj Speedway R420), and an emotion recognition sensor (e.g., Intel RealSense D435) in the living room.

[0272] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0273] 3. Data analysis: The server generates storage suggestions using AI algorithms (e.g., TensorFlow), and the emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0274] 4. Proposal notification: The terminal notifies the user of this proposal.

[0275] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0276] 6. Recording information: The server records this information in a database (e.g. MySQL).

[0277] Example 2: Product search

[0278] 1. Search request: The user asks the app, "Where is the remote?"

[0279] 2. Database search: The server searches the database (e.g., PostgreSQL) and obtains the location information, such as "The remote control is in the drawer of the table in the living room."

[0280] 3. Notification: The device notifies the user of this information.

[0281] Examples of prompt statements

[0282] Below are some examples of prompt sentences to input into the generative AI model.

[0283] "Please suggest the optimal storage method for the living room. Please generate an efficient storage method using furniture layout data and item location data collected by camera sensors and RFID sensors, and user emotion data collected by emotion recognition sensors."

[0284] This system allows for efficient storage method suggestions, item management, and searching, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0285] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0286] Step 1: Sensor installation and data collection

[0287] The server initializes the camera sensor (e.g., Nest Cam), RFID sensor (e.g., Impinj Speedway R420), and emotion recognition sensor (e.g., RealSense D435). It collects data from the sensors in real time. As input, it receives image data from the camera sensor, item ID data from the RFID sensor, and facial expression and voice tone data from the emotion recognition sensor. The output is a set of these sensor data.

[0288] Step 2: Data Preprocessing

[0289] The server preprocesses the collected sensor data. It crops or filters the image data from the camera sensor to remove noise, decodes the data from the RFID sensor and converts it into an item ID, and analyzes the data obtained from the emotion recognition sensor to determine the user's emotional state. The inputs are the image data, RFID data, and emotion data collected in step 1. The outputs are the cropped image data, decoded item ID, and analyzed emotion data.

[0290] Step 3: Analysis by AI algorithm

[0291] The server inputs the cropped image data, decoded item IDs, and analyzed emotion data into an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes these data and generates an optimal storage method. The input is the preprocessed data output from Step 2. The output is a specific storage suggestion (e.g., "Store your textbooks on the shelf next to the sofa in the living room").

[0292] Step 4: Adjusting suggestions with the emotion engine

[0293] The server adjusts the storage suggestions based on the AI ​​algorithm and the analyzed emotional data. For example, if the user is feeling stressed, it will prepare an easy storage method that makes it easy to access. The input is the storage suggestions generated in step 3 and the emotional data. The output is the storage suggestions adjusted according to the user's emotional state.

[0294] Step 5: Proposal Notification and Interface

[0295] The device receives the adjusted storage suggestions from the server. The suggestions are displayed as pop-up notifications or on the UI via a smartphone or tablet app. The input is notification data containing the adjusted storage suggestions. The output is the specific suggestions presented to the user.

[0296] Step 6: Record and manage product information

[0297] The user stores the item in a designated location and enters the storage location into the app. The device sends this information to the server, which records it in the database. The input is the item storage location data entered by the user into the app. The output is the updated database information.

[0298] Step 7: Search for and notify items

[0299] When a user asks the app, "Where's the remote?", the server searches the database and retrieves the location information of the item. The device notifies the user of this information. The input is the search query for the item. The output is the location information of the found item.

[0300] (Application example 2)

[0301] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] Inventory management and product placement in physical stores require a lot of time and effort, and poses a significant workload for store staff. Furthermore, store staff stress and busyness can affect inventory management and product placement, making it difficult to carry out their work efficiently. In order for store staff to work efficiently under these circumstances, they need to be able to grasp the situation in real time and receive accurate instructions. However, conventional systems have not been able to fully achieve this.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0304] In this invention, the server includes: means for collecting data from sensors installed in the living space; artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods; means for recognizing the user's emotions and reflecting them in the suggestions; means for the user to input the storage locations of items; database means for recording and managing the input storage locations and item information; means for searching for and notifying the location of items based on the database information; and means for notifying and displaying the suggestions on a device such as smart glasses or a tablet. This allows for optimal inventory management and product placement suggestions to be made in real time, even in physical stores, reducing the workload of store clerks and enabling more efficient work. Furthermore, by providing optimal suggestions based on the emotional state of store clerks, stress and fatigue can be reduced, improving work efficiency.

[0305] "Sensors installed in residential spaces" refers to devices, such as camera sensors and RFID sensors, installed to collect information about items and the environment within a specific space.

[0306] "Means of collecting data from sensors" refers to methods, equipment, and software for collecting data obtained from camera sensors, RFID sensors, etc. in real time or periodically.

[0307] "Artificial intelligence means for analyzing data and suggesting storage solutions" refers to AI algorithms and related software used to process and analyze collected data and generate and suggest optimal storage solutions and layouts.

[0308] "Means of recognizing users' emotions and reflecting them in proposals" refers to methods, equipment, and software that use sensors to obtain emotional data such as users' facial expressions and tone of voice, analyze it, and reflect it in proposal content.

[0309] "Means for inputting the storage location of an item" refers to an interface, device, or software that allows a user to input where an item has been stored into the system.

[0310] "Database means" refers to a database system and related software for recording and managing collected data and entered information.

[0311] "Means for searching for and notifying the location of an item based on database information" refers to a method, device, or software for searching for the location of an item using information recorded in a database and notifying the user of the results.

[0312] "Means for notifying and displaying suggestion content on devices such as smart glasses and tablets" refers to applications or software for displaying suggestions and notification content generated by AI on electronic devices such as smart glasses, tablets, and smartphones.

[0313] The "inventory management and product placement proposal system for brick-and-mortar stores" of the present invention is a system that uses in-store sensors and artificial intelligence to propose efficient inventory management and product placement, thereby reducing the workload of store staff. Specific embodiments are described below.

[0314] Sensor installation and data collection

[0315] The server collects data in real time from camera sensors and RFID sensors installed in the store. The camera sensors capture image data of the shelves, and the RFID sensors read the ID data of each product. The emotion engine also uses sensors to detect the tone of voice and facial expressions of store staff. This data is sent to the server for preprocessing in preparation for the next analysis step.

[0316] Data analysis and proposal generation

[0317] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow). The AI ​​algorithm analyzes this data and generates optimal inventory management and product placement suggestions. For example, it may make a suggestion such as, "Popular products need to be replenished." The emotion engine analyzes the emotional data of the store clerk and adjusts the suggestions based on this information. For example, if the store clerk is feeling stressed, it may make suggestions that are easy to implement.

[0318] Suggestion notifications and interface

[0319] The device (e.g., smart glasses or tablet) generates suggestions based on the AI ​​algorithm and emotion engine and notifies the salesperson. The suggestions are provided via the device's app as pop-up notifications or highlighted in the UI. The salesperson can then use this information to replenish or rearrange products.

[0320] Recording and managing inventory information

[0321] The user (store clerk) inputs product placement and inventory data into the terminal. For example, they input "add new product to shelf." This information is sent to the server and recorded in the database.

[0322] Search and Notifications

[0323] When a user wants to check on a particular product or inventory, they can ask the device, "Where is product X?" The server searches the database to get the location information and sends it to the device, which then reports this information to a store associate and, in some cases, provides an easy-to-read map display of the location.

[0324] By implementing the above, store inventory management and product placement can be carried out efficiently, and suggestions can be made based on the emotional state of store staff, reducing their workload and improving the efficiency of store operations.

[0325] Examples of prompt statements

[0326] "Write a program that generates optimal suggestions to optimize product placement in a store using data collected from camera sensors and RFID sensors, as well as the emotional data of store clerks. In particular, make easy suggestions when store clerks are stressed, and suggest the optimal placement method when they are not."

[0327] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0328] Step 1:

[0329] The server collects data in real time from camera sensors and RFID sensors installed in the store. Specifically, the camera sensors take images of the shelves, and the RFID sensors read the product ID data. This collected data is sent to the server. The inputs are image data and ID data, and the output is saved in an integrated format.

[0330] Step 2:

[0331] The server preprocesses the collected sensor data. Image data is preprocessed using image processing libraries such as OpenCV to remove noise and adjust resolution. Meanwhile, RFID data is checked for duplication and errors and formatted as clean data. The input is the collected data, and the output is preprocessed clean data.

[0332] Step 3:

[0333] The server inputs the preprocessed data into an AI algorithm. For example, TensorFlow is used to detect available shelf space and product counts from image data. This processing generates optimal inventory management and product placement recommendations. The input is the preprocessed data, and the output is the recommendations.

[0334] Step 4:

[0335] The server uses an emotion engine to analyze the clerk's emotional data. It uses an emotion recognition API (e.g., Microsoft® Azure® Cognitive Services) to analyze the clerk's tone of voice and facial expressions, and reflects the results in the recommendations. The input is emotional data obtained from the sensor, and the output is the emotion analysis results.

[0336] Step 5:

[0337] The server integrates the results of the AI ​​algorithm and emotion engine to generate the final proposal. For example, if the store clerk is busy, it will suggest a replenishment that can be easily implemented, and if not, it will suggest the optimal placement. The input is the results of the AI ​​algorithm and emotion engine, and the integrated proposal is obtained as the output.

[0338] Step 6:

[0339] The device notifies and displays the generated proposal on the smart glasses or tablet. Specifically, a pop-up notification is displayed in the app or a highlight is displayed on the UI. The input is the final proposal, and the proposal is notified to the store clerk as the output.

[0340] Step 7:

[0341] The user (store clerk) replenishes and arranges products according to the suggestions displayed on the terminal. Specific actions include moving or replenishing products to the suggested locations. The input is the suggestions, and the output is the completed replenishment and new arrangement information.

[0342] Step 8:

[0343] The user inputs product placement and inventory data into the device. For example, they input information such as "add a new product to the shelf" into the app. The input is product placement information, and the output is updated database information.

[0344] Step 9:

[0345] The server searches for the location of the item based on the database information and notifies the user. When a user checks a specific product or inventory, the server searches the database to obtain location information and sends the results to the device. The input is a search query, and the output is location information.

[0346] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0347] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0348] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0349] [Second embodiment]

[0350] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0351] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0352] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0353] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0354] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0355] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0356] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0357] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0358] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0359] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0360] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0361] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0362] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to propose efficient storage methods and support the management and retrieval of items. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0363] Overall system configuration

[0364] Sensor installation and data collection

[0365] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0366] Data analysis and storage proposals

[0367] The server preprocesses the collected sensor data and passes it as input parameters to an AI algorithm, which then analyzes the data and generates the optimal storage method. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0368] Suggestion notifications and interface

[0369] The device notifies the user of storage suggestions generated by the AI ​​algorithm. The suggestions are displayed as pop-up notifications or highlighted on the UI via the smartphone or tablet app. The user then uses these suggestions to store items in the designated locations.

[0370] Recording and managing product information

[0371] When a user puts an item away, they input the storage location into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0372] Search and Notifications

[0373] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0374] Specific system operation examples

[0375] Example 1: Organizing your living room

[0376] 1. Sensor installation: Install a camera sensor and an RFID sensor in the living room.

[0377] 2. Data collection: The server collects furniture layout and item location data from sensors.

[0378] 3. Data analysis: The server uses an AI algorithm to analyze, for example, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0379] 4. Proposal notification: The terminal notifies the user of this proposal.

[0380] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0381] 6. Recording information: The server records this information in a database.

[0382] Example 2: Product search

[0383] 1. Search request: The user asks the app, "Where is the remote?"

[0384] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0385] 3. Notification: The device notifies the user of this information.

[0386] This system allows for efficient storage method suggestions, item management, and item searching, improving the quality of life for users.

[0387] The processing flow will be explained below.

[0388] Step 1:

[0389] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server.

[0390] Step 2:

[0391] The server preprocesses the collected data: in the case of image data, it uses object recognition algorithms to detect items and extract location information, and in the case of RFID data, it determines the item's ID and location.

[0392] Step 3:

[0393] The server inputs the pre-processed data into an artificial intelligence algorithm to analyze the living space. The analysis results in an efficient storage solution. Specifically, it calculates where and how to store items to maximize space utilization.

[0394] Step 4:

[0395] The device notifies the user of the storage suggestions received from the server, and the suggestions are presented to the user via a smartphone or tablet app as pop-up notifications or highlighted on the interface.

[0396] Step 5:

[0397] The user stores the items in the designated locations according to the suggested storage method, for example, by following instructions such as "store the books in the bookshelf in the living room."

[0398] Step 6:

[0399] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0400] Step 7:

[0401] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0402] Step 8:

[0403] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0404] Step 9:

[0405] The server searches the database and obtains the location information of the relevant item. For example, the server obtains location information such as "The remote control is in the drawer of the table in the living room."

[0406] Step 10:

[0407] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0408] Step 11:

[0409] The server collects user feedback and updates the AI ​​algorithm, allowing it to learn and improve the accuracy of future suggestions.

[0410] Example 1

[0411] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0412] Efficient storage and retrieval of items is a crucial issue in modern living spaces, but it is difficult to manage a variety of items easily while making effective use of limited space. Conventional systems require users to manage items manually, which takes time and effort, and users often forget where they have stored them. Furthermore, there is a lack of efficient storage suggestions, making it difficult to make optimal use of living space.

[0413] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0414] In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors, artificial intelligence means for analyzing the preprocessed data and proposing storage methods, means for notifying the user of the proposed storage methods, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, and means for searching for and notifying the location of items based on the database information, thereby enabling efficient storage method suggestions, item management, and rapid search.

[0415] "Sensors" are devices used to collect location information about objects and furniture in living spaces, including camera sensors and RFID sensors.

[0416] "Preprocessing" is the process of shaping or filtering data collected from sensors to make it easier to analyze.

[0417] "Artificial intelligence means" refers to technology for generating and proposing efficient storage methods based on preprocessed data, and machine learning algorithms are included in this section.

[0418] The "notification means" is a method for informing the user of the storage method suggestions generated by the server, and is mainly an application on a smartphone or tablet.

[0419] "Database means" means a system for recording and managing the location information of items and other related information entered by users.

[0420] A "search means" is a system that has the ability to identify the location of an item based on information in a database and notify the user of that information.

[0421] The "Smart Organizing Support AI System" of this invention is a system that uses sensors and artificial intelligence installed in living spaces to suggest efficient storage methods and assist with managing and retrieving items. The system aims to provide optimal storage methods for making effective use of limited space and reduce the effort required for users to search for items.

[0422] Overall system configuration

[0423] Sensor installation and data collection

[0424] The server collects real-time data from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0425] Data Preprocessing

[0426] The server pre-processes the collected raw data: image data from the camera sensors is analyzed using image processing algorithms to identify furniture layout and object locations, and RFID data is decoded to extract item identification information.

[0427] Organizing and storage proposal generation

[0428] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0429] proposal notification

[0430] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The user uses this information to store the items in the specified location.

[0431] Recording the location of items

[0432] When a user places an item in a designated location, they input the location information into the app. For example, they might input "Put new books in the bookshelf in the living room." This information is then sent back to the server and recorded in the database.

[0433] Item location search and notification

[0434] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map.

[0435] Specific system operation examples

[0436] Living room organization

[0437] 1. A user installs a camera sensor on the ceiling of their living room and places RFID sensors on key furniture pieces.

[0438] 2. The server periodically collects image data from the camera sensor and item ID data from the RFID sensor.

[0439] 3. The server uses image analysis algorithms to analyze the furniture layout and extract item identification information from the RFID data.

[0440] 4. The server uses an AI algorithm to analyze the situation and generate a suggestion: "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0441] 5. The device will then display a pop-up notification suggesting to the user that "it would be better to store magazines in the storage box next to the sofa."

[0442] 6. The user enters "Put the magazine in the box next to the sofa" into the app, and the item's location information is sent to the server.

[0443] 7. The server records the transmitted information in a database.

[0444] Product Search

[0445] 1. The user types into the app, "Where's the remote?"

[0446] 2. The server searches the database and obtains the location information: "The remote control is in the table drawer in the living room."

[0447] 3. Your device will display a pop-up notification with this information, possibly showing your location along with a floor plan of your living room.

[0448] Example prompts for generative AI models

[0449] "Design an AI algorithm that suggests efficient ways to store items."

[0450] "Please explain the program flow for building an inventory management system using data collected from sensors."

[0451] This system allows users to organize their living space and manage their belongings simply and efficiently, thereby improving the quality of their lives.

[0452] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0453] Step 1: Sensor installation and data collection

[0454] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. The input is raw data from the camera sensors and RFID sensors, and the output is the captured image data and ID information.

[0455] Step 2: Preprocessing the data

[0456] The server preprocesses the collected raw data. Specifically, it analyzes the image data using image processing algorithms to identify furniture layout and object locations. It also decodes the RFID data to extract item identification information. The inputs are raw data from camera sensors and RFID sensors, and the output is preprocessed location and identification information.

[0457] Step 3: Generate organization and storage proposals

[0458] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room. The inputs are preprocessed location information and identification information, and the output is a specific storage suggestion.

[0459] Step 4: Proposal Notification

[0460] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The input is the storage suggestions from the server, and the notified suggestion information is obtained as output.

[0461] Step 5: Record the location of the item

[0462] When a user stores an item in a designated location, they input the location information into the app. For example, they might enter "Store new books in the living room bookshelf." This information is sent back to the server and recorded in the database. The input is the location information from the user, and the output is the information recorded in the database.

[0463] Step 6: Search for and notify location of item

[0464] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map. The input is the user's question, and the output is the location information of the search results.

[0465] By clarifying what data processing and calculations are performed based on the input data at each step and what output is ultimately obtained, it becomes easier to understand the flow of the entire system.

[0466] (Application example 1)

[0467] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0468] Logistics centers handle a wide variety of products in large quantities, so they need to propose optimal storage methods and efficiently search for them. With conventional systems, product placement within the warehouse was inefficient, and locating products took a lot of time and effort. This resulted in reduced operational efficiency and a serious labor shortage.

[0469] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0470] In this invention, the server includes means for collecting data from sensors installed in the living space, artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods, means for users to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching and notifying the location of items based on the database information, and means for supporting the storage and search of packages at the logistics center, thereby enabling efficient management of packages and rapid search at the logistics center.

[0471] A "sensor" is a device for collecting data from a physical space, and includes, for example, a camera sensor or an RFID sensor.

[0472] "Means for collecting data" refers to a system that has the function of collecting data obtained from sensors and sending it to a server.

[0473] "Artificial intelligence means" refers to algorithms or software that analyze collected data and make recommendations or decisions according to specific purposes.

[0474] "Means for users to input the storage location of items" refers to an interface where users provide information about storage locations and items, which has the function of transmitting the information to a server.

[0475] "Database means" refers to a storage device and management system for recording and managing input storage location and item information.

[0476] "Means for searching for and notifying the location of an item" refers to a system that has the function of searching for the location of an item based on information recorded in a database and notifying the user of the results.

[0477] "Means to support the storage and retrieval of goods in a logistics center" refers to functions and systems that assist in the optimal placement and rapid retrieval of goods within a logistics center.

[0478] The "Smart Organizing Support AI System" based on this invention supports efficient storage and retrieval of packages in logistics centers. This system analyzes data collected from sensors and proposes optimal storage methods to improve the efficiency of package management and retrieval. A specific form of the system that realizes this invention will be described.

[0479] Sensor installation and data collection

[0480] The sensors installed in the distribution center include camera sensors and RFID sensors. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to a server via the Internet.

[0481] Data analysis and storage proposals

[0482] The server preprocesses the collected sensor data and stores it in a database. The preprocessed data is then input into a generative AI model, where an algorithm is applied to suggest optimal storage locations. This results in specific recommendations, such as "Place the new product on shelf B-2." This analysis is performed using machine learning algorithms using TensorFlow.

[0483] Suggestion notifications and interface

[0484] The server then sends the generated storage suggestions to smartphones and tablets. Warehouse staff receive the suggestions via pop-up notifications or highlighted displays on the UI, and store the items in the designated locations accordingly.

[0485] Recording and managing product information

[0486] After storing an item in a designated location, the user enters the information into the terminal. For example, they might enter "Store item X on shelf B-2." This information is sent to the server and recorded in a database. This allows for centralized management of item information within the logistics center and allows for real-time updates.

[0487] Search and Notifications

[0488] When a user wants to find a specific item, they enter a search query into their device. For example, they ask, "Where is item X?" The server searches the database and retrieves the location of the item. The search results are sent to the device and displayed to the user. In some cases, a visual map display is also displayed.

[0489] Specific examples

[0490] 1. Data Collection

[0491] Camera sensors are installed inside the warehouse to capture images of shelf layout and inventory status.

[0492] RFID sensors are used to scan the RFID tags on each product to collect information.

[0493] 2. Data Analysis

[0494] For example, the AI ​​will make suggestions such as, "This shelf has a weight limit, so move heavy items to another shelf."

[0495] 3. Proposal Notice

[0496] You receive a pop-up notification on your smartphone app saying, "Please place the new product on shelf B-2."

[0497] 4. Information Records

[0498] The warehouse staff enters "Place product X on shelf B-2" into the app and records it on the server.

[0499] 5. Product Search

[0500] Type into the app, "Where is product X?"

[0501] The server notifies, "Product X is on shelf B-2."

[0502] Prompt Sentence Examples

[0503] "Generate helpful storage suggestions for users in your distribution center. This should take into account the weight, size, and current inventory of an item and suggest the best storage location. For example, suggest placing heavy items on lower shelves and lighter items on higher shelves."

[0504] This enables the system to efficiently manage and quickly search for packages within the logistics center.

[0505] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0506] Step 1: Data collection

[0507] The server collects data from camera sensors and RFID sensors installed in the distribution center. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to the server via the Internet. The input is real-time data from the sensors, and the output is data sent to the server.

[0508] Step 2: Data Preprocessing

[0509] The server preprocesses the collected image data and RFID tag data. This process involves noise removal, data format conversion, and extraction of necessary data. The input is the collected sensor data, and the output is the preprocessed data.

[0510] Step 3: Data analysis

[0511] The server inputs the preprocessed data into a generative AI model and performs analysis to suggest optimal storage locations. For example, TensorFlow is used to consider the weight and size of items and current inventory status to suggest the optimal shelf. The input is the preprocessed data, and the output is storage suggestions generated by the AI.

[0512] Step 4: Notification of storage proposal

[0513] The server notifies the generated storage suggestions to the smartphone or tablet device, which then communicates the specific suggestions to the user through pop-up notifications or highlighting on the UI. The input is the suggestions generated by the AI, and the output is the notified suggestions.

[0514] Step 5: Recording product information

[0515] The user stores the items in the designated locations according to the suggestions and enters the information into the terminal. The terminal sends this information to the server, which records it in the database. The input is the item information sent by the user, and the output is the information recorded in the database.

[0516] Step 6: Search for items

[0517] When a user wants to find a specific item, they input a search query into their device. The server searches the database and obtains the location information of the corresponding item. The obtained information is sent to the device and notified to the user. The input is the user's search query, and the output is the location information of the item.

[0518] These processing steps result in efficient package management and rapid retrieval at the logistics center.

[0519] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0520] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with the management and retrieval of items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. This system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0521] Overall system configuration

[0522] Sensor installation and data collection

[0523] The server collects data in real time from camera and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server. The emotion engine also uses sensors to detect the user's tone of voice, facial expressions, and gestures.

[0524] Data analysis and storage proposals

[0525] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm. The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts its suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0526] Suggestion notifications and interface

[0527] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[0528] Recording and managing product information

[0529] When a user puts an item away, they input the storage location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0530] Search and Notifications

[0531] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0532] Specific system operation examples

[0533] Example 1: Organizing your living room

[0534] 1. Sensor installation: Install a camera sensor, RFID sensor, and emotion recognition sensor in the living room.

[0535] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0536] 3. Data analysis: The server generates storage suggestions using AI algorithms, and an emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0537] 4. Proposal notification: The terminal notifies the user of this proposal.

[0538] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0539] 6. Recording information: The server records this information in a database.

[0540] Example 2: Product search

[0541] 1. Search request: The user asks the app, "Where is the remote?"

[0542] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0543] 3. Notification: The device notifies the user of this information.

[0544] This system allows for efficient storage method suggestions, item management, and search, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0545] The processing flow will be explained below.

[0546] Step 1:

[0547] The server collects data in real time from camera sensors, RFID sensors, and emotion recognition sensors installed in the living space. The camera sensors capture image data of the room and furniture layout, while the RFID sensors collect ID data of items. The emotion recognition sensors analyze the user's tone of voice, facial expressions, and gestures to obtain emotional data.

[0548] Step 2:

[0549] The server preprocesses the collected data: for image data, it uses object recognition algorithms to detect items and extract location information; for RFID data, it identifies item IDs and locations; and for emotional data, it analyzes emotional states from voice, facial expressions, and gestures.

[0550] Step 3:

[0551] The server inputs the preprocessed data into an artificial intelligence algorithm to analyze the space usage. As a result of the analysis, it generates an efficient storage method. For example, it makes a specific suggestion such as "It would be good to store textbooks on the shelf next to the sofa in the living room." At this time, an emotion engine adjusts the suggestion based on the user's emotional state. For example, if the user is feeling stressed, it suggests a storage method that is simple and easy to access.

[0552] Step 4:

[0553] The device then notifies the user of the storage suggestions received from the server. The suggestions are displayed as notifications or highlighted on the interface via the smartphone or tablet app. For example, the device may notify the user that "it would be convenient to store textbooks on the shelf on the right side of the living room."

[0554] Step 5:

[0555] The user stores items in the designated location according to the storage method suggested by the user. For example, "store books in the bookshelf in the living room."

[0556] Step 6:

[0557] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0558] Step 7:

[0559] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0560] Step 8:

[0561] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0562] Step 9:

[0563] The server searches the database and obtains the location information of the relevant item. For example, the location information is obtained in the form of "The remote control is in the drawer of the table in the living room."

[0564] Step 10:

[0565] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0566] Step 11:

[0567] The server collects user feedback and updates the AI ​​algorithm and emotion engine, allowing it to learn and improve the accuracy of future suggestions.

[0568] Example 2

[0569] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0570] Currently, there is a lack of efficient methods for storing and managing items in living spaces. Furthermore, it is often time-consuming for users to search for items, making it difficult to effectively utilize living space. Furthermore, existing systems do not provide user-friendly experiences because they do not take into account the user's emotional state. A new system is needed to solve these problems.

[0571] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors and passing it to an artificial intelligence algorithm, means for notifying the user of the storage method generated by the artificial intelligence algorithm, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching for and notifying the location of items based on the database information, and means for analyzing emotional data and making suggestions according to the user's emotional state. This enables efficient storage method suggestions, item management and search, and user-friendly suggestions according to the user's emotional state.

[0572] "Living space" refers to the space and environment in which people live their daily lives.

[0573] A "sensor" refers to a device that detects a physical phenomenon and outputs that information as an electrical signal.

[0574] "Data preprocessing" refers to processes such as trimming, filtering, and decoding of raw data collected from sensors to make it easier to analyze.

[0575] "Artificial intelligence algorithms" refers to mathematical and statistical methods for analyzing data to perform specific tasks, and specifically includes machine learning and deep learning techniques.

[0576] "Storage method" refers to the techniques and procedures for arranging items efficiently and optimally.

[0577] "Notification" refers to the act of notifying a specific device or user of certain information.

[0578] "Input means" refers to the method or device by which a user provides information to a system.

[0579] A "database" refers to a system that systematically records and manages information and enables it to be searched and retrieved as needed.

[0580] "Emotional data" refers to information that indicates a user's emotional state, such as tone of voice, facial expressions, and gestures.

[0581] "User-friendly" refers to a design concept that allows users to operate intuitively and comfortably.

[0582] The "Smart Organizing Support AI System" of this invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with managing and searching for items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items.

[0583] Overall system configuration

[0584] Sensor installation and data collection

[0585] The server collects data in real time from camera sensors (e.g., Nest Cam) and RFID sensors (e.g., Impinj Speedway R420) installed in the living space. Image data captured by the camera sensors and item ID data acquired by RFID sensors are sent to the server. In addition, sensors used for the emotion engine (e.g., Intrinsic RealSense D435) detect the user's tone of voice, facial expressions, and gestures.

[0586] Data analysis and storage proposals

[0587] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts the suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0588] Suggestion notifications and interface

[0589] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app (e.g., iOS or Android app) as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[0590] Recording and managing product information

[0591] When a user puts an item away, they input the location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in a database (e.g., MySQL or PostgreSQL).

[0592] Search and Notifications

[0593] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0594] Specific system operation examples

[0595] Example 1: Organizing your living room

[0596] 1. Sensor installation: Install a camera sensor (e.g., Nest Cam), an RFID sensor (e.g., Impinj Speedway R420), and an emotion recognition sensor (e.g., Intel RealSense D435) in the living room.

[0597] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0598] 3. Data analysis: The server generates storage suggestions using AI algorithms (e.g., TensorFlow), and the emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0599] 4. Proposal notification: The terminal notifies the user of this proposal.

[0600] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0601] 6. Recording information: The server records this information in a database (e.g. MySQL).

[0602] Example 2: Product search

[0603] 1. Search request: The user asks the app, "Where is the remote?"

[0604] 2. Database search: The server searches the database (e.g., PostgreSQL) and obtains the location information, such as "The remote control is in the drawer of the table in the living room."

[0605] 3. Notification: The device notifies the user of this information.

[0606] Examples of prompt statements

[0607] Below are some examples of prompt sentences to input into the generative AI model.

[0608] "Please suggest the optimal storage method for the living room. Please generate an efficient storage method using furniture layout data and item location data collected by camera sensors and RFID sensors, and user emotion data collected by emotion recognition sensors."

[0609] This system allows for efficient storage method suggestions, item management, and searching, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0610] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0611] Step 1: Sensor installation and data collection

[0612] The server initializes the camera sensor (e.g., Nest Cam), RFID sensor (e.g., Impinj Speedway R420), and emotion recognition sensor (e.g., RealSense D435). It collects data from the sensors in real time. As input, it receives image data from the camera sensor, item ID data from the RFID sensor, and facial expression and voice tone data from the emotion recognition sensor. The output is a set of these sensor data.

[0613] Step 2: Data Preprocessing

[0614] The server preprocesses the collected sensor data. It crops or filters the image data from the camera sensor to remove noise, decodes the data from the RFID sensor and converts it into an item ID, and analyzes the data obtained from the emotion recognition sensor to determine the user's emotional state. The inputs are the image data, RFID data, and emotion data collected in step 1. The outputs are the cropped image data, decoded item ID, and analyzed emotion data.

[0615] Step 3: Analysis by AI algorithm

[0616] The server inputs the cropped image data, decoded item IDs, and analyzed emotion data into an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes these data and generates an optimal storage method. The input is the preprocessed data output from Step 2. The output is a specific storage suggestion (e.g., "Store your textbooks on the shelf next to the sofa in the living room").

[0617] Step 4: Adjusting suggestions with the emotion engine

[0618] The server adjusts the storage suggestions based on the AI ​​algorithm and the analyzed emotional data. For example, if the user is feeling stressed, it will prepare an easy storage method that makes it easy to access. The input is the storage suggestions generated in step 3 and the emotional data. The output is the storage suggestions adjusted according to the user's emotional state.

[0619] Step 5: Proposal Notification and Interface

[0620] The device receives the adjusted storage suggestions from the server. The suggestions are displayed as pop-up notifications or on the UI via a smartphone or tablet app. The input is notification data containing the adjusted storage suggestions. The output is the specific suggestions presented to the user.

[0621] Step 6: Record and manage product information

[0622] The user stores the item in a designated location and enters the storage location into the app. The device sends this information to the server, which records it in the database. The input is the item storage location data entered by the user into the app. The output is the updated database information.

[0623] Step 7: Search for and notify items

[0624] When a user asks the app, "Where's the remote?", the server searches the database and retrieves the location information of the item. The device notifies the user of this information. The input is the search query for the item. The output is the location information of the found item.

[0625] (Application example 2)

[0626] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] Inventory management and product placement in physical stores require a lot of time and effort, and poses a significant workload for store staff. Furthermore, store staff stress and busyness can affect inventory management and product placement, making it difficult to carry out their work efficiently. In order for store staff to work efficiently under these circumstances, they need to be able to grasp the situation in real time and receive accurate instructions. However, conventional systems have not been able to fully achieve this.

[0628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0629] In this invention, the server includes: means for collecting data from sensors installed in the living space; artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods; means for recognizing the user's emotions and reflecting them in the suggestions; means for the user to input the storage locations of items; database means for recording and managing the input storage locations and item information; means for searching for and notifying the location of items based on the database information; and means for notifying and displaying the suggestions on a device such as smart glasses or a tablet. This allows for optimal inventory management and product placement suggestions to be made in real time, even in physical stores, reducing the workload of store clerks and enabling more efficient work. Furthermore, by providing optimal suggestions based on the emotional state of store clerks, stress and fatigue can be reduced, improving work efficiency.

[0630] "Sensors installed in residential spaces" refers to devices, such as camera sensors and RFID sensors, installed to collect information about items and the environment within a specific space.

[0631] "Means of collecting data from sensors" refers to methods, equipment, and software for collecting data obtained from camera sensors, RFID sensors, etc. in real time or periodically.

[0632] "Artificial intelligence means for analyzing data and suggesting storage solutions" refers to AI algorithms and related software used to process and analyze collected data and generate and suggest optimal storage solutions and layouts.

[0633] "Means of recognizing users' emotions and reflecting them in proposals" refers to methods, equipment, and software that use sensors to obtain emotional data such as users' facial expressions and tone of voice, analyze it, and reflect it in proposal content.

[0634] "Means for inputting the storage location of an item" refers to an interface, device, or software that allows a user to input where an item has been stored into the system.

[0635] "Database means" refers to a database system and related software for recording and managing collected data and entered information.

[0636] "Means for searching for and notifying the location of an item based on database information" refers to a method, device, or software for searching for the location of an item using information recorded in a database and notifying the user of the results.

[0637] "Means for notifying and displaying suggestion content on devices such as smart glasses and tablets" refers to applications or software for displaying suggestions and notification content generated by AI on electronic devices such as smart glasses, tablets, and smartphones.

[0638] The "inventory management and product placement proposal system for brick-and-mortar stores" of the present invention is a system that uses in-store sensors and artificial intelligence to propose efficient inventory management and product placement, thereby reducing the workload of store staff. Specific embodiments are described below.

[0639] Sensor installation and data collection

[0640] The server collects data in real time from camera sensors and RFID sensors installed in the store. The camera sensors capture image data of the shelves, and the RFID sensors read the ID data of each product. The emotion engine also uses sensors to detect the tone of voice and facial expressions of store staff. This data is sent to the server for preprocessing in preparation for the next analysis step.

[0641] Data analysis and proposal generation

[0642] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow). The AI ​​algorithm analyzes this data and generates optimal inventory management and product placement suggestions. For example, it may make a suggestion such as, "Popular products need to be replenished." The emotion engine analyzes the emotional data of the store clerk and adjusts the suggestions based on this information. For example, if the store clerk is feeling stressed, it may make suggestions that are easy to implement.

[0643] Suggestion notifications and interface

[0644] The device (e.g., smart glasses or tablet) generates suggestions based on the AI ​​algorithm and emotion engine and notifies the salesperson. The suggestions are provided via the device's app as pop-up notifications or highlighted in the UI. The salesperson can then use this information to replenish or rearrange products.

[0645] Recording and managing inventory information

[0646] The user (store clerk) inputs product placement and inventory data into the terminal. For example, they input "add new product to shelf." This information is sent to the server and recorded in the database.

[0647] Search and Notifications

[0648] When a user wants to check on a particular product or inventory, they can ask the device, "Where is product X?" The server searches the database to get the location information and sends it to the device, which then reports this information to a store associate and, in some cases, provides an easy-to-read map display of the location.

[0649] By implementing the above, store inventory management and product placement can be carried out efficiently, and suggestions can be made based on the emotional state of store staff, reducing their workload and improving the efficiency of store operations.

[0650] Examples of prompt statements

[0651] "Write a program that generates optimal suggestions to optimize product placement in a store using data collected from camera sensors and RFID sensors, as well as the emotional data of store clerks. In particular, make easy suggestions when store clerks are stressed, and suggest the optimal placement method when they are not."

[0652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0653] Step 1:

[0654] The server collects data in real time from camera sensors and RFID sensors installed in the store. Specifically, the camera sensors take images of the shelves, and the RFID sensors read the product ID data. This collected data is sent to the server. The inputs are image data and ID data, and the output is saved in an integrated format.

[0655] Step 2:

[0656] The server preprocesses the collected sensor data. Image data is preprocessed using image processing libraries such as OpenCV to remove noise and adjust resolution. Meanwhile, RFID data is checked for duplication and errors and formatted as clean data. The input is the collected data, and the output is preprocessed clean data.

[0657] Step 3:

[0658] The server inputs the preprocessed data into an AI algorithm. For example, TensorFlow is used to detect available shelf space and product counts from image data. This processing generates optimal inventory management and product placement recommendations. The input is the preprocessed data, and the output is the recommendations.

[0659] Step 4:

[0660] The server uses an emotion engine to analyze the clerk's emotional data. It uses an emotion recognition API (e.g., Microsoft Azure Cognitive Services) to analyze the clerk's tone of voice and facial expressions, and reflects the results in the recommendations. The input is emotional data obtained from the sensor, and the output is the emotion analysis result.

[0661] Step 5:

[0662] The server integrates the results of the AI ​​algorithm and emotion engine to generate the final proposal. For example, if the store clerk is busy, it will suggest a replenishment that can be easily implemented, and if not, it will suggest the optimal placement. The input is the results of the AI ​​algorithm and emotion engine, and the integrated proposal is obtained as the output.

[0663] Step 6:

[0664] The device notifies and displays the generated proposal on the smart glasses or tablet. Specifically, a pop-up notification is displayed in the app or a highlight is displayed on the UI. The input is the final proposal, and the proposal is notified to the store clerk as the output.

[0665] Step 7:

[0666] The user (store clerk) replenishes and arranges products according to the suggestions displayed on the terminal. Specific actions include moving or replenishing products to the suggested locations. The input is the suggestions, and the output is the completed replenishment and new arrangement information.

[0667] Step 8:

[0668] The user inputs product placement and inventory data into the device. For example, they input information such as "add a new product to the shelf" into the app. The input is product placement information, and the output is updated database information.

[0669] Step 9:

[0670] The server searches for the location of the item based on the database information and notifies the user. When a user checks a specific product or inventory, the server searches the database to obtain location information and sends the results to the device. The input is a search query, and the output is location information.

[0671] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0672] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0673] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0674] [Third embodiment]

[0675] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0676] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0677] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0678] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0679] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0680] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0681] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0682] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0683] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0684] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0685] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0686] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0687] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to propose efficient storage methods and support the management and retrieval of items. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0688] Overall system configuration

[0689] Sensor installation and data collection

[0690] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0691] Data analysis and storage proposals

[0692] The server preprocesses the collected sensor data and passes it as input parameters to an AI algorithm, which then analyzes the data and generates the optimal storage method. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0693] Suggestion notifications and interface

[0694] The device notifies the user of storage suggestions generated by the AI ​​algorithm. The suggestions are displayed as pop-up notifications or highlighted on the UI via the smartphone or tablet app. The user then uses these suggestions to store items in the designated locations.

[0695] Recording and managing product information

[0696] When a user puts an item away, they input the storage location into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0697] Search and Notifications

[0698] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0699] Specific system operation examples

[0700] Example 1: Organizing your living room

[0701] 1. Sensor installation: Install a camera sensor and an RFID sensor in the living room.

[0702] 2. Data collection: The server collects furniture layout and item location data from sensors.

[0703] 3. Data analysis: The server uses an AI algorithm to analyze, for example, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0704] 4. Proposal notification: The terminal notifies the user of this proposal.

[0705] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0706] 6. Recording information: The server records this information in a database.

[0707] Example 2: Product search

[0708] 1. Search request: The user asks the app, "Where is the remote?"

[0709] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0710] 3. Notification: The device notifies the user of this information.

[0711] This system allows for efficient storage method suggestions, item management, and item searching, improving the quality of life for users.

[0712] The processing flow will be explained below.

[0713] Step 1:

[0714] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server.

[0715] Step 2:

[0716] The server preprocesses the collected data: in the case of image data, it uses object recognition algorithms to detect items and extract location information, and in the case of RFID data, it determines the item's ID and location.

[0717] Step 3:

[0718] The server inputs the pre-processed data into an artificial intelligence algorithm to analyze the living space. The analysis results in an efficient storage solution. Specifically, it calculates where and how to store items to maximize space utilization.

[0719] Step 4:

[0720] The device notifies the user of the storage suggestions received from the server, and the suggestions are presented to the user via a smartphone or tablet app as pop-up notifications or highlighted on the interface.

[0721] Step 5:

[0722] The user stores the items in the designated locations according to the suggested storage method, for example, by following instructions such as "store the books in the bookshelf in the living room."

[0723] Step 6:

[0724] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0725] Step 7:

[0726] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0727] Step 8:

[0728] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0729] Step 9:

[0730] The server searches the database and obtains the location information of the relevant item. For example, the server obtains location information such as "The remote control is in the drawer of the table in the living room."

[0731] Step 10:

[0732] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0733] Step 11:

[0734] The server collects user feedback and updates the AI ​​algorithm, allowing it to learn and improve the accuracy of future suggestions.

[0735] Example 1

[0736] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0737] Efficient storage and retrieval of items is a crucial issue in modern living spaces, but it is difficult to manage a variety of items easily while making effective use of limited space. Conventional systems require users to manage items manually, which takes time and effort, and users often forget where they have stored them. Furthermore, there is a lack of efficient storage suggestions, making it difficult to make optimal use of living space.

[0738] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0739] In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors, artificial intelligence means for analyzing the preprocessed data and proposing storage methods, means for notifying the user of the proposed storage methods, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, and means for searching for and notifying the location of items based on the database information, thereby enabling efficient storage method suggestions, item management, and rapid search.

[0740] "Sensors" are devices used to collect location information about objects and furniture in living spaces, including camera sensors and RFID sensors.

[0741] "Preprocessing" is the process of shaping or filtering data collected from sensors to make it easier to analyze.

[0742] "Artificial intelligence means" refers to technology for generating and proposing efficient storage methods based on preprocessed data, and machine learning algorithms are included in this section.

[0743] The "notification means" is a method for informing the user of the storage method suggestions generated by the server, and is mainly an application on a smartphone or tablet.

[0744] "Database means" means a system for recording and managing the location information of items and other related information entered by users.

[0745] A "search means" is a system that has the ability to identify the location of an item based on information in a database and notify the user of that information.

[0746] The "Smart Organizing Support AI System" of this invention is a system that uses sensors and artificial intelligence installed in living spaces to suggest efficient storage methods and assist with managing and retrieving items. The system aims to provide optimal storage methods for making effective use of limited space and reduce the effort required for users to search for items.

[0747] Overall system configuration

[0748] Sensor installation and data collection

[0749] The server collects real-time data from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[0750] Data Preprocessing

[0751] The server pre-processes the collected raw data: image data from the camera sensors is analyzed using image processing algorithms to identify furniture layout and object locations, and RFID data is decoded to extract item identification information.

[0752] Organizing and storage proposal generation

[0753] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[0754] proposal notification

[0755] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The user uses this information to store the items in the specified location.

[0756] Recording the location of items

[0757] When a user places an item in a designated location, they input the location information into the app. For example, they might input "Put new books in the bookshelf in the living room." This information is then sent back to the server and recorded in the database.

[0758] Item location search and notification

[0759] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map.

[0760] Specific system operation examples

[0761] Living room organization

[0762] 1. A user installs a camera sensor on the ceiling of their living room and places RFID sensors on key furniture pieces.

[0763] 2. The server periodically collects image data from the camera sensor and item ID data from the RFID sensor.

[0764] 3. The server uses image analysis algorithms to analyze the furniture layout and extract item identification information from the RFID data.

[0765] 4. The server uses an AI algorithm to analyze the situation and generate a suggestion: "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0766] 5. The device will then display a pop-up notification suggesting to the user that "it would be better to store magazines in the storage box next to the sofa."

[0767] 6. The user enters "Put the magazine in the box next to the sofa" into the app, and the item's location information is sent to the server.

[0768] 7. The server records the transmitted information in a database.

[0769] Product Search

[0770] 1. The user types into the app, "Where's the remote?"

[0771] 2. The server searches the database and obtains the location information: "The remote control is in the table drawer in the living room."

[0772] 3. Your device will display a pop-up notification with this information, possibly showing your location along with a floor plan of your living room.

[0773] Example prompts for generative AI models

[0774] "Design an AI algorithm that suggests efficient ways to store items."

[0775] "Please explain the program flow for building an inventory management system using data collected from sensors."

[0776] This system allows users to organize their living space and manage their belongings simply and efficiently, thereby improving the quality of their lives.

[0777] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0778] Step 1: Sensor installation and data collection

[0779] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. The input is raw data from the camera sensors and RFID sensors, and the output is the captured image data and ID information.

[0780] Step 2: Preprocessing the data

[0781] The server preprocesses the collected raw data. Specifically, it analyzes the image data using image processing algorithms to identify furniture layout and object locations. It also decodes the RFID data to extract item identification information. The inputs are raw data from camera sensors and RFID sensors, and the output is preprocessed location and identification information.

[0782] Step 3: Generate organization and storage proposals

[0783] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room. The inputs are preprocessed location information and identification information, and the output is a specific storage suggestion.

[0784] Step 4: Proposal Notification

[0785] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The input is the storage suggestions from the server, and the notified suggestion information is obtained as output.

[0786] Step 5: Record the location of the item

[0787] When a user stores an item in a designated location, they input the location information into the app. For example, they might enter "Store new books in the living room bookshelf." This information is sent back to the server and recorded in the database. The input is the location information from the user, and the output is the information recorded in the database.

[0788] Step 6: Search for and notify location of item

[0789] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map. The input is the user's question, and the output is the location information of the search results.

[0790] By clarifying what data processing and calculations are performed based on the input data at each step and what output is ultimately obtained, it becomes easier to understand the flow of the entire system.

[0791] (Application example 1)

[0792] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0793] Logistics centers handle a wide variety of products in large quantities, so they need to propose optimal storage methods and efficiently search for them. With conventional systems, product placement within the warehouse was inefficient, and locating products took a lot of time and effort. This resulted in reduced operational efficiency and a serious labor shortage.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0795] In this invention, the server includes means for collecting data from sensors installed in the living space, artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods, means for users to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching and notifying the location of items based on the database information, and means for supporting the storage and search of packages at the logistics center, thereby enabling efficient management of packages and rapid search at the logistics center.

[0796] A "sensor" is a device for collecting data from a physical space, and includes, for example, a camera sensor or an RFID sensor.

[0797] "Means for collecting data" refers to a system that has the function of collecting data obtained from sensors and sending it to a server.

[0798] "Artificial intelligence means" refers to algorithms or software that analyze collected data and make recommendations or decisions according to specific purposes.

[0799] "Means for users to input the storage location of items" refers to an interface where users provide information about storage locations and items, which has the function of transmitting the information to a server.

[0800] "Database means" refers to a storage device and management system for recording and managing input storage location and item information.

[0801] "Means for searching for and notifying the location of an item" refers to a system that has the function of searching for the location of an item based on information recorded in a database and notifying the user of the results.

[0802] "Means to support the storage and retrieval of goods in a logistics center" refers to functions and systems that assist in the optimal placement and rapid retrieval of goods within a logistics center.

[0803] The "Smart Organizing Support AI System" based on this invention supports efficient storage and retrieval of packages in logistics centers. This system analyzes data collected from sensors and proposes optimal storage methods to improve the efficiency of package management and retrieval. A specific form of the system that realizes this invention will be described.

[0804] Sensor installation and data collection

[0805] The sensors installed in the distribution center include camera sensors and RFID sensors. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to a server via the Internet.

[0806] Data analysis and storage proposals

[0807] The server preprocesses the collected sensor data and stores it in a database. The preprocessed data is then input into a generative AI model, where an algorithm is applied to suggest optimal storage locations. This results in specific recommendations, such as "Place the new product on shelf B-2." This analysis is performed using machine learning algorithms using TensorFlow.

[0808] Suggestion notifications and interface

[0809] The server then sends the generated storage suggestions to smartphones and tablets. Warehouse staff receive the suggestions via pop-up notifications or highlighted displays on the UI, and store the items in the designated locations accordingly.

[0810] Recording and managing product information

[0811] After storing an item in a designated location, the user enters the information into the terminal. For example, they might enter "Store item X on shelf B-2." This information is sent to the server and recorded in a database. This allows for centralized management of item information within the logistics center and allows for real-time updates.

[0812] Search and Notifications

[0813] When a user wants to find a specific item, they enter a search query into their device. For example, they ask, "Where is item X?" The server searches the database and retrieves the location of the item. The search results are sent to the device and displayed to the user. In some cases, a visual map display is also displayed.

[0814] Specific examples

[0815] 1. Data Collection

[0816] Camera sensors are installed inside the warehouse to capture images of shelf layout and inventory status.

[0817] RFID sensors are used to scan the RFID tags on each product to collect information.

[0818] 2. Data Analysis

[0819] For example, the AI ​​will make suggestions such as, "This shelf has a weight limit, so move heavy items to another shelf."

[0820] 3. Proposal Notice

[0821] You receive a pop-up notification on your smartphone app saying, "Please place the new product on shelf B-2."

[0822] 4. Information Records

[0823] The warehouse staff enters "Place product X on shelf B-2" into the app and records it on the server.

[0824] 5. Product Search

[0825] Type into the app, "Where is product X?"

[0826] The server notifies, "Product X is on shelf B-2."

[0827] Prompt Sentence Examples

[0828] "Generate helpful storage suggestions for users in your distribution center. This should take into account the weight, size, and current inventory of an item and suggest the best storage location. For example, suggest placing heavy items on lower shelves and lighter items on higher shelves."

[0829] This enables the system to efficiently manage and quickly search for packages within the logistics center.

[0830] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0831] Step 1: Data collection

[0832] The server collects data from camera sensors and RFID sensors installed in the distribution center. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to the server via the Internet. The input is real-time data from the sensors, and the output is data sent to the server.

[0833] Step 2: Data Preprocessing

[0834] The server preprocesses the collected image data and RFID tag data. This process involves noise removal, data format conversion, and extraction of necessary data. The input is the collected sensor data, and the output is the preprocessed data.

[0835] Step 3: Data analysis

[0836] The server inputs the preprocessed data into a generative AI model and performs analysis to suggest optimal storage locations. For example, TensorFlow is used to consider the weight and size of items and current inventory status to suggest the optimal shelf. The input is the preprocessed data, and the output is storage suggestions generated by the AI.

[0837] Step 4: Notification of storage proposal

[0838] The server notifies the generated storage suggestions to the smartphone or tablet device, which then communicates the specific suggestions to the user through pop-up notifications or highlighting on the UI. The input is the suggestions generated by the AI, and the output is the notified suggestions.

[0839] Step 5: Recording product information

[0840] The user stores the items in the designated locations according to the suggestions and enters the information into the terminal. The terminal sends this information to the server, which records it in the database. The input is the item information sent by the user, and the output is the information recorded in the database.

[0841] Step 6: Search for items

[0842] When a user wants to find a specific item, they input a search query into their device. The server searches the database and obtains the location information of the corresponding item. The obtained information is sent to the device and notified to the user. The input is the user's search query, and the output is the location information of the item.

[0843] These processing steps result in efficient package management and rapid retrieval at the logistics center.

[0844] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0845] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with the management and retrieval of items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. This system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[0846] Overall system configuration

[0847] Sensor installation and data collection

[0848] The server collects data in real time from camera and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server. The emotion engine also uses sensors to detect the user's tone of voice, facial expressions, and gestures.

[0849] Data analysis and storage proposals

[0850] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm. The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts its suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0851] Suggestion notifications and interface

[0852] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[0853] Recording and managing product information

[0854] When a user puts an item away, they input the storage location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[0855] Search and Notifications

[0856] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0857] Specific system operation examples

[0858] Example 1: Organizing your living room

[0859] 1. Sensor installation: Install a camera sensor, RFID sensor, and emotion recognition sensor in the living room.

[0860] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0861] 3. Data analysis: The server generates storage suggestions using AI algorithms, and an emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0862] 4. Proposal notification: The terminal notifies the user of this proposal.

[0863] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0864] 6. Recording information: The server records this information in a database.

[0865] Example 2: Product search

[0866] 1. Search request: The user asks the app, "Where is the remote?"

[0867] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[0868] 3. Notification: The device notifies the user of this information.

[0869] This system allows for efficient storage method suggestions, item management, and search, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0870] The processing flow will be explained below.

[0871] Step 1:

[0872] The server collects data in real time from camera sensors, RFID sensors, and emotion recognition sensors installed in the living space. The camera sensors capture image data of the room and furniture layout, while the RFID sensors collect ID data of items. The emotion recognition sensors analyze the user's tone of voice, facial expressions, and gestures to obtain emotional data.

[0873] Step 2:

[0874] The server preprocesses the collected data: for image data, it uses object recognition algorithms to detect items and extract location information; for RFID data, it identifies item IDs and locations; and for emotional data, it analyzes emotional states from voice, facial expressions, and gestures.

[0875] Step 3:

[0876] The server inputs the preprocessed data into an artificial intelligence algorithm to analyze the space usage. As a result of the analysis, it generates an efficient storage method. For example, it makes a specific suggestion such as "It would be good to store textbooks on the shelf next to the sofa in the living room." At this time, an emotion engine adjusts the suggestion based on the user's emotional state. For example, if the user is feeling stressed, it suggests a storage method that is simple and easy to access.

[0877] Step 4:

[0878] The device then notifies the user of the storage suggestions received from the server. The suggestions are displayed as notifications or highlighted on the interface via the smartphone or tablet app. For example, the device may notify the user that "it would be convenient to store textbooks on the shelf on the right side of the living room."

[0879] Step 5:

[0880] The user stores items in the designated location according to the storage method suggested by the user. For example, "store books in the bookshelf in the living room."

[0881] Step 6:

[0882] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[0883] Step 7:

[0884] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[0885] Step 8:

[0886] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[0887] Step 9:

[0888] The server searches the database and obtains the location information of the relevant item. For example, the location information is obtained in the form of "The remote control is in the drawer of the table in the living room."

[0889] Step 10:

[0890] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[0891] Step 11:

[0892] The server collects user feedback and updates the AI ​​algorithm and emotion engine, allowing it to learn and improve the accuracy of future suggestions.

[0893] Example 2

[0894] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0895] Currently, there is a lack of efficient methods for storing and managing items in living spaces. Furthermore, it is often time-consuming for users to search for items, making it difficult to effectively utilize living space. Furthermore, existing systems do not provide user-friendly experiences because they do not take into account the user's emotional state. A new system is needed to solve these problems.

[0896] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors and passing it to an artificial intelligence algorithm, means for notifying the user of the storage method generated by the artificial intelligence algorithm, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching for and notifying the location of items based on the database information, and means for analyzing emotional data and making suggestions according to the user's emotional state. This enables efficient storage method suggestions, item management and search, and user-friendly suggestions according to the user's emotional state.

[0897] "Living space" refers to the space and environment in which people live their daily lives.

[0898] A "sensor" refers to a device that detects a physical phenomenon and outputs that information as an electrical signal.

[0899] "Data preprocessing" refers to processes such as trimming, filtering, and decoding of raw data collected from sensors to make it easier to analyze.

[0900] "Artificial intelligence algorithms" refers to mathematical and statistical methods for analyzing data to perform specific tasks, and specifically includes machine learning and deep learning techniques.

[0901] "Storage method" refers to the techniques and procedures for arranging items efficiently and optimally.

[0902] "Notification" refers to the act of notifying a specific device or user of certain information.

[0903] "Input means" refers to the method or device by which a user provides information to a system.

[0904] A "database" refers to a system that systematically records and manages information and enables it to be searched and retrieved as needed.

[0905] "Emotional data" refers to information that indicates a user's emotional state, such as tone of voice, facial expressions, and gestures.

[0906] "User-friendly" refers to a design concept that allows users to operate intuitively and comfortably.

[0907] The "Smart Organizing Support AI System" of this invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with managing and searching for items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items.

[0908] Overall system configuration

[0909] Sensor installation and data collection

[0910] The server collects data in real time from camera sensors (e.g., Nest Cam) and RFID sensors (e.g., Impinj Speedway R420) installed in the living space. Image data captured by the camera sensors and item ID data acquired by RFID sensors are sent to the server. In addition, sensors used for the emotion engine (e.g., Intrinsic RealSense D435) detect the user's tone of voice, facial expressions, and gestures.

[0911] Data analysis and storage proposals

[0912] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts the suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[0913] Suggestion notifications and interface

[0914] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app (e.g., iOS or Android app) as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[0915] Recording and managing product information

[0916] When a user puts an item away, they input the location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in a database (e.g., MySQL or PostgreSQL).

[0917] Search and Notifications

[0918] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[0919] Specific system operation examples

[0920] Example 1: Organizing your living room

[0921] 1. Sensor installation: Install a camera sensor (e.g., Nest Cam), an RFID sensor (e.g., Impinj Speedway R420), and an emotion recognition sensor (e.g., Intel RealSense D435) in the living room.

[0922] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[0923] 3. Data analysis: The server generates storage suggestions using AI algorithms (e.g., TensorFlow), and the emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[0924] 4. Proposal notification: The terminal notifies the user of this proposal.

[0925] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[0926] 6. Recording information: The server records this information in a database (e.g. MySQL).

[0927] Example 2: Product search

[0928] 1. Search request: The user asks the app, "Where is the remote?"

[0929] 2. Database search: The server searches the database (e.g., PostgreSQL) and obtains the location information, such as "The remote control is in the drawer of the table in the living room."

[0930] 3. Notification: The device notifies the user of this information.

[0931] Examples of prompt statements

[0932] Below are some examples of prompt sentences to input into the generative AI model.

[0933] "Please suggest the optimal storage method for the living room. Please generate an efficient storage method using furniture layout data and item location data collected by camera sensors and RFID sensors, and user emotion data collected by emotion recognition sensors."

[0934] This system allows for efficient storage method suggestions, item management, and searching, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[0935] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0936] Step 1: Sensor installation and data collection

[0937] The server initializes the camera sensor (e.g., Nest Cam), RFID sensor (e.g., Impinj Speedway R420), and emotion recognition sensor (e.g., RealSense D435). It collects data from the sensors in real time. As input, it receives image data from the camera sensor, item ID data from the RFID sensor, and facial expression and voice tone data from the emotion recognition sensor. The output is a set of these sensor data.

[0938] Step 2: Data Preprocessing

[0939] The server preprocesses the collected sensor data. It crops or filters the image data from the camera sensor to remove noise, decodes the data from the RFID sensor and converts it into an item ID, and analyzes the data obtained from the emotion recognition sensor to determine the user's emotional state. The inputs are the image data, RFID data, and emotion data collected in step 1. The outputs are the cropped image data, decoded item ID, and analyzed emotion data.

[0940] Step 3: Analysis by AI algorithm

[0941] The server inputs the cropped image data, decoded item IDs, and analyzed emotion data into an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes these data and generates an optimal storage method. The input is the preprocessed data output from Step 2. The output is a specific storage suggestion (e.g., "Store your textbooks on the shelf next to the sofa in the living room").

[0942] Step 4: Adjusting suggestions with the emotion engine

[0943] The server adjusts the storage suggestions based on the AI ​​algorithm and the analyzed emotional data. For example, if the user is feeling stressed, it will prepare an easy storage method that makes it easy to access. The input is the storage suggestions generated in step 3 and the emotional data. The output is the storage suggestions adjusted according to the user's emotional state.

[0944] Step 5: Proposal Notification and Interface

[0945] The device receives the adjusted storage suggestions from the server. The suggestions are displayed as pop-up notifications or on the UI via a smartphone or tablet app. The input is notification data containing the adjusted storage suggestions. The output is the specific suggestions presented to the user.

[0946] Step 6: Record and manage product information

[0947] The user stores the item in a designated location and enters the storage location into the app. The device sends this information to the server, which records it in the database. The input is the item storage location data entered by the user into the app. The output is the updated database information.

[0948] Step 7: Search for and notify items

[0949] When a user asks the app, "Where's the remote?", the server searches the database and retrieves the location information of the item. The device notifies the user of this information. The input is the search query for the item. The output is the location information of the found item.

[0950] (Application example 2)

[0951] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0952] Inventory management and product placement in physical stores require a lot of time and effort, and poses a significant workload for store staff. Furthermore, store staff stress and busyness can affect inventory management and product placement, making it difficult to carry out their work efficiently. In order for store staff to work efficiently under these circumstances, they need to be able to grasp the situation in real time and receive accurate instructions. However, conventional systems have not been able to fully achieve this.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0954] In this invention, the server includes: means for collecting data from sensors installed in the living space; artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods; means for recognizing the user's emotions and reflecting them in the suggestions; means for the user to input the storage locations of items; database means for recording and managing the input storage locations and item information; means for searching for and notifying the location of items based on the database information; and means for notifying and displaying the suggestions on a device such as smart glasses or a tablet. This allows for optimal inventory management and product placement suggestions to be made in real time, even in physical stores, reducing the workload of store clerks and enabling more efficient work. Furthermore, by providing optimal suggestions based on the emotional state of store clerks, stress and fatigue can be reduced, improving work efficiency.

[0955] "Sensors installed in residential spaces" refers to devices, such as camera sensors and RFID sensors, installed to collect information about items and the environment within a specific space.

[0956] "Means of collecting data from sensors" refers to methods, equipment, and software for collecting data obtained from camera sensors, RFID sensors, etc. in real time or periodically.

[0957] "Artificial intelligence means for analyzing data and suggesting storage solutions" refers to AI algorithms and related software used to process and analyze collected data and generate and suggest optimal storage solutions and layouts.

[0958] "Means of recognizing users' emotions and reflecting them in proposals" refers to methods, equipment, and software that use sensors to obtain emotional data such as users' facial expressions and tone of voice, analyze it, and reflect it in proposal content.

[0959] "Means for inputting the storage location of an item" refers to an interface, device, or software that allows a user to input where an item has been stored into the system.

[0960] "Database means" refers to a database system and related software for recording and managing collected data and entered information.

[0961] "Means for searching for and notifying the location of an item based on database information" refers to a method, device, or software for searching for the location of an item using information recorded in a database and notifying the user of the results.

[0962] "Means for notifying and displaying suggestion content on devices such as smart glasses and tablets" refers to applications or software for displaying suggestions and notification content generated by AI on electronic devices such as smart glasses, tablets, and smartphones.

[0963] The "inventory management and product placement proposal system for brick-and-mortar stores" of the present invention is a system that uses in-store sensors and artificial intelligence to propose efficient inventory management and product placement, thereby reducing the workload of store staff. Specific embodiments are described below.

[0964] Sensor installation and data collection

[0965] The server collects data in real time from camera sensors and RFID sensors installed in the store. The camera sensors capture image data of the shelves, and the RFID sensors read the ID data of each product. The emotion engine also uses sensors to detect the tone of voice and facial expressions of store staff. This data is sent to the server for preprocessing in preparation for the next analysis step.

[0966] Data analysis and proposal generation

[0967] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow). The AI ​​algorithm analyzes this data and generates optimal inventory management and product placement suggestions. For example, it may make a suggestion such as, "Popular products need to be replenished." The emotion engine analyzes the emotional data of the store clerk and adjusts the suggestions based on this information. For example, if the store clerk is feeling stressed, it may make suggestions that are easy to implement.

[0968] Suggestion notifications and interface

[0969] The device (e.g., smart glasses or tablet) generates suggestions based on the AI ​​algorithm and emotion engine and notifies the salesperson. The suggestions are provided via the device's app as pop-up notifications or highlighted in the UI. The salesperson can then use this information to replenish or rearrange products.

[0970] Recording and managing inventory information

[0971] The user (store clerk) inputs product placement and inventory data into the terminal. For example, they input "add new product to shelf." This information is sent to the server and recorded in the database.

[0972] Search and Notifications

[0973] When a user wants to check on a particular product or inventory, they can ask the device, "Where is product X?" The server searches the database to get the location information and sends it to the device, which then reports this information to a store associate and, in some cases, provides an easy-to-read map display of the location.

[0974] By implementing the above, store inventory management and product placement can be carried out efficiently, and suggestions can be made based on the emotional state of store staff, reducing their workload and improving the efficiency of store operations.

[0975] Examples of prompt statements

[0976] "Write a program that generates optimal suggestions to optimize product placement in a store using data collected from camera sensors and RFID sensors, as well as the emotional data of store clerks. In particular, make easy suggestions when store clerks are stressed, and suggest the optimal placement method when they are not."

[0977] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0978] Step 1:

[0979] The server collects data in real time from camera sensors and RFID sensors installed in the store. Specifically, the camera sensors take images of the shelves, and the RFID sensors read the product ID data. This collected data is sent to the server. The inputs are image data and ID data, and the output is saved in an integrated format.

[0980] Step 2:

[0981] The server preprocesses the collected sensor data. Image data is preprocessed using image processing libraries such as OpenCV to remove noise and adjust resolution. Meanwhile, RFID data is checked for duplication and errors and formatted as clean data. The input is the collected data, and the output is preprocessed clean data.

[0982] Step 3:

[0983] The server inputs the preprocessed data into an AI algorithm. For example, TensorFlow is used to detect available shelf space and product counts from image data. This processing generates optimal inventory management and product placement recommendations. The input is the preprocessed data, and the output is the recommendations.

[0984] Step 4:

[0985] The server uses an emotion engine to analyze the clerk's emotional data. It uses an emotion recognition API (e.g., Microsoft Azure Cognitive Services) to analyze the clerk's tone of voice and facial expressions, and reflects the results in the recommendations. The input is emotional data obtained from the sensor, and the output is the emotion analysis result.

[0986] Step 5:

[0987] The server integrates the results of the AI ​​algorithm and emotion engine to generate the final proposal. For example, if the store clerk is busy, it will suggest a replenishment that can be easily implemented, and if not, it will suggest the optimal placement. The input is the results of the AI ​​algorithm and emotion engine, and the integrated proposal is obtained as the output.

[0988] Step 6:

[0989] The device notifies and displays the generated proposal on the smart glasses or tablet. Specifically, a pop-up notification is displayed in the app or a highlight is displayed on the UI. The input is the final proposal, and the proposal is notified to the store clerk as the output.

[0990] Step 7:

[0991] The user (store clerk) replenishes and arranges products according to the suggestions displayed on the terminal. Specific actions include moving or replenishing products to the suggested locations. The input is the suggestions, and the output is the completed replenishment and new arrangement information.

[0992] Step 8:

[0993] The user inputs product placement and inventory data into the device. For example, they input information such as "add a new product to the shelf" into the app. The input is product placement information, and the output is updated database information.

[0994] Step 9:

[0995] The server searches for the location of the item based on the database information and notifies the user. When a user checks a specific product or inventory, the server searches the database to obtain location information and sends the results to the device. The input is a search query, and the output is location information.

[0996] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0997] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0998] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0999] [Fourth embodiment]

[1000] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1001] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1002] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1003] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1004] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1005] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1006] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1007] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1008] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1009] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1010] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1011] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1012] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1013] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to propose efficient storage methods and support the management and retrieval of items. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[1014] Overall system configuration

[1015] Sensor installation and data collection

[1016] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[1017] Data analysis and storage proposals

[1018] The server preprocesses the collected sensor data and passes it as input parameters to an AI algorithm, which then analyzes the data and generates the optimal storage method. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[1019] Suggestion notifications and interface

[1020] The device notifies the user of storage suggestions generated by the AI ​​algorithm. The suggestions are displayed as pop-up notifications or highlighted on the UI via the smartphone or tablet app. The user then uses these suggestions to store items in the designated locations.

[1021] Recording and managing product information

[1022] When a user puts an item away, they input the storage location into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[1023] Search and Notifications

[1024] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[1025] Specific system operation examples

[1026] Example 1: Organizing your living room

[1027] 1. Sensor installation: Install a camera sensor and an RFID sensor in the living room.

[1028] 2. Data collection: The server collects furniture layout and item location data from sensors.

[1029] 3. Data analysis: The server uses an AI algorithm to analyze, for example, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[1030] 4. Proposal notification: The terminal notifies the user of this proposal.

[1031] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[1032] 6. Recording information: The server records this information in a database.

[1033] Example 2: Product search

[1034] 1. Search request: The user asks the app, "Where is the remote?"

[1035] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[1036] 3. Notification: The device notifies the user of this information.

[1037] This system allows for efficient storage method suggestions, item management, and item searching, improving the quality of life for users.

[1038] The processing flow will be explained below.

[1039] Step 1:

[1040] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server.

[1041] Step 2:

[1042] The server preprocesses the collected data: in the case of image data, it uses object recognition algorithms to detect items and extract location information, and in the case of RFID data, it determines the item's ID and location.

[1043] Step 3:

[1044] The server inputs the pre-processed data into an artificial intelligence algorithm to analyze the living space. The analysis results in an efficient storage solution. Specifically, it calculates where and how to store items to maximize space utilization.

[1045] Step 4:

[1046] The device notifies the user of the storage suggestions received from the server, and the suggestions are presented to the user via a smartphone or tablet app as pop-up notifications or highlighted on the interface.

[1047] Step 5:

[1048] The user stores the items in the designated locations according to the suggested storage method, for example, by following instructions such as "store the books in the bookshelf in the living room."

[1049] Step 6:

[1050] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[1051] Step 7:

[1052] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[1053] Step 8:

[1054] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[1055] Step 9:

[1056] The server searches the database and obtains the location information of the relevant item. For example, the server obtains location information such as "The remote control is in the drawer of the table in the living room."

[1057] Step 10:

[1058] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[1059] Step 11:

[1060] The server collects user feedback and updates the AI ​​algorithm, allowing it to learn and improve the accuracy of future suggestions.

[1061] Example 1

[1062] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1063] Efficient storage and retrieval of items is a crucial issue in modern living spaces, but it is difficult to manage a variety of items easily while making effective use of limited space. Conventional systems require users to manage items manually, which takes time and effort, and users often forget where they have stored them. Furthermore, there is a lack of efficient storage suggestions, making it difficult to make optimal use of living space.

[1064] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1065] In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors, artificial intelligence means for analyzing the preprocessed data and proposing storage methods, means for notifying the user of the proposed storage methods, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, and means for searching for and notifying the location of items based on the database information, thereby enabling efficient storage method suggestions, item management, and rapid search.

[1066] "Sensors" are devices used to collect location information about objects and furniture in living spaces, including camera sensors and RFID sensors.

[1067] "Preprocessing" is the process of shaping or filtering data collected from sensors to make it easier to analyze.

[1068] "Artificial intelligence means" refers to technology for generating and proposing efficient storage methods based on preprocessed data, and machine learning algorithms are included in this section.

[1069] The "notification means" is a method for informing the user of the storage method suggestions generated by the server, and is mainly an application on a smartphone or tablet.

[1070] "Database means" means a system for recording and managing the location information of items and other related information entered by users.

[1071] A "search means" is a system that has the ability to identify the location of an item based on information in a database and notify the user of that information.

[1072] The "Smart Organizing Support AI System" of this invention is a system that uses sensors and artificial intelligence installed in living spaces to suggest efficient storage methods and assist with managing and retrieving items. The system aims to provide optimal storage methods for making effective use of limited space and reduce the effort required for users to search for items.

[1073] Overall system configuration

[1074] Sensor installation and data collection

[1075] The server collects real-time data from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. This data is sent from the sensors to the server via the Internet.

[1076] Data Preprocessing

[1077] The server pre-processes the collected raw data: image data from the camera sensors is analyzed using image processing algorithms to identify furniture layout and object locations, and RFID data is decoded to extract item identification information.

[1078] Organizing and storage proposal generation

[1079] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room.

[1080] proposal notification

[1081] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The user uses this information to store the items in the specified location.

[1082] Recording the location of items

[1083] When a user places an item in a designated location, they input the location information into the app. For example, they might input "Put new books in the bookshelf in the living room." This information is then sent back to the server and recorded in the database.

[1084] Item location search and notification

[1085] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map.

[1086] Specific system operation examples

[1087] Living room organization

[1088] 1. A user installs a camera sensor on the ceiling of their living room and places RFID sensors on key furniture pieces.

[1089] 2. The server periodically collects image data from the camera sensor and item ID data from the RFID sensor.

[1090] 3. The server uses image analysis algorithms to analyze the furniture layout and extract item identification information from the RFID data.

[1091] 4. The server uses an AI algorithm to analyze the situation and generate a suggestion: "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[1092] 5. The device will then display a pop-up notification suggesting to the user that "it would be better to store magazines in the storage box next to the sofa."

[1093] 6. The user enters "Put the magazine in the box next to the sofa" into the app, and the item's location information is sent to the server.

[1094] 7. The server records the transmitted information in a database.

[1095] Product Search

[1096] 1. The user types into the app, "Where's the remote?"

[1097] 2. The server searches the database and obtains the location information: "The remote control is in the table drawer in the living room."

[1098] 3. Your device will display a pop-up notification with this information, possibly showing your location along with a floor plan of your living room.

[1099] Example prompts for generative AI models

[1100] "Design an AI algorithm that suggests efficient ways to store items."

[1101] "Please explain the program flow for building an inventory management system using data collected from sensors."

[1102] This system allows users to organize their living space and manage their belongings simply and efficiently, thereby improving the quality of their lives.

[1103] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1104] Step 1: Sensor installation and data collection

[1105] The server collects data in real time from camera sensors and RFID sensors installed in the living space. The camera sensors capture image data of the entire room and the furniture layout, while the RFID sensors read the ID information of items. The input is raw data from the camera sensors and RFID sensors, and the output is the captured image data and ID information.

[1106] Step 2: Preprocessing the data

[1107] The server preprocesses the collected raw data. Specifically, it analyzes the image data using image processing algorithms to identify furniture layout and object locations. It also decodes the RFID data to extract item identification information. The inputs are raw data from camera sensors and RFID sensors, and the output is preprocessed location and identification information.

[1108] Step 3: Generate organization and storage proposals

[1109] The server runs an artificial intelligence algorithm based on the preprocessed data. The AI ​​generates efficient storage methods and suggests specific locations for storing items. For example, it might suggest that textbooks should be stored on the shelf next to the sofa in the living room. The inputs are preprocessed location information and identification information, and the output is a specific storage suggestion.

[1110] Step 4: Proposal Notification

[1111] The device notifies the user of the storage suggestions received from the server. Specifically, the user is notified by a pop-up notification through the smartphone or tablet app, or by highlighting the suggestions in the app's UI. The input is the storage suggestions from the server, and the notified suggestion information is obtained as output.

[1112] Step 5: Record the location of the item

[1113] When a user stores an item in a designated location, they input the location information into the app. For example, they might enter "Store new books in the living room bookshelf." This information is sent back to the server and recorded in the database. The input is the location information from the user, and the output is the information recorded in the database.

[1114] Step 6: Search for and notify location of item

[1115] The user uses the app to input a question such as "Where is the remote control?" The server receives this question and searches the database to obtain the item's location information. The obtained information is sent to the device, which notifies the user, and in some cases, the location is displayed on an easy-to-read map. The input is the user's question, and the output is the location information of the search results.

[1116] By clarifying what data processing and calculations are performed based on the input data at each step and what output is ultimately obtained, it becomes easier to understand the flow of the entire system.

[1117] (Application example 1)

[1118] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1119] Logistics centers handle a wide variety of products in large quantities, so they need to propose optimal storage methods and efficiently search for them. With conventional systems, product placement within the warehouse was inefficient, and locating products took a lot of time and effort. This resulted in reduced operational efficiency and a serious labor shortage.

[1120] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1121] In this invention, the server includes means for collecting data from sensors installed in the living space, artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods, means for users to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching and notifying the location of items based on the database information, and means for supporting the storage and search of packages at the logistics center, thereby enabling efficient management of packages and rapid search at the logistics center.

[1122] A "sensor" is a device for collecting data from a physical space, and includes, for example, a camera sensor or an RFID sensor.

[1123] "Means for collecting data" refers to a system that has the function of collecting data obtained from sensors and sending it to a server.

[1124] "Artificial intelligence means" refers to algorithms or software that analyze collected data and make recommendations or decisions according to specific purposes.

[1125] "Means for users to input the storage location of items" refers to an interface where users provide information about storage locations and items, which has the function of transmitting the information to a server.

[1126] "Database means" refers to a storage device and management system for recording and managing input storage location and item information.

[1127] "Means for searching for and notifying the location of an item" refers to a system that has the function of searching for the location of an item based on information recorded in a database and notifying the user of the results.

[1128] "Means to support the storage and retrieval of goods in a logistics center" refers to functions and systems that assist in the optimal placement and rapid retrieval of goods within a logistics center.

[1129] The "Smart Organizing Support AI System" based on this invention supports efficient storage and retrieval of packages in logistics centers. This system analyzes data collected from sensors and proposes optimal storage methods to improve the efficiency of package management and retrieval. A specific form of the system that realizes this invention will be described.

[1130] Sensor installation and data collection

[1131] The sensors installed in the distribution center include camera sensors and RFID sensors. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to a server via the Internet.

[1132] Data analysis and storage proposals

[1133] The server preprocesses the collected sensor data and stores it in a database. The preprocessed data is then input into a generative AI model, where an algorithm is applied to suggest optimal storage locations. This results in specific recommendations, such as "Place the new product on shelf B-2." This analysis is performed using machine learning algorithms using TensorFlow.

[1134] Suggestion notifications and interface

[1135] The server then sends the generated storage suggestions to smartphones and tablets. Warehouse staff receive the suggestions via pop-up notifications or highlighted displays on the UI, and store the items in the designated locations accordingly.

[1136] Recording and managing product information

[1137] After storing an item in a designated location, the user enters the information into the terminal. For example, they might enter "Store item X on shelf B-2." This information is sent to the server and recorded in a database. This allows for centralized management of item information within the logistics center and allows for real-time updates.

[1138] Search and Notifications

[1139] When a user wants to find a specific item, they enter a search query into their device. For example, they ask, "Where is item X?" The server searches the database and retrieves the location of the item. The search results are sent to the device and displayed to the user. In some cases, a visual map display is also displayed.

[1140] Specific examples

[1141] 1. Data Collection

[1142] Camera sensors are installed inside the warehouse to capture images of shelf layout and inventory status.

[1143] RFID sensors are used to scan the RFID tags on each product to collect information.

[1144] 2. Data Analysis

[1145] For example, the AI ​​will make suggestions such as, "This shelf has a weight limit, so move heavy items to another shelf."

[1146] 3. Proposal Notice

[1147] You receive a pop-up notification on your smartphone app saying, "Please place the new product on shelf B-2."

[1148] 4. Information Records

[1149] The warehouse staff enters "Place product X on shelf B-2" into the app and records it on the server.

[1150] 5. Product Search

[1151] Type into the app, "Where is product X?"

[1152] The server notifies, "Product X is on shelf B-2."

[1153] Prompt Sentence Examples

[1154] "Generate helpful storage suggestions for users in your distribution center. This should take into account the weight, size, and current inventory of an item and suggest the best storage location. For example, suggest placing heavy items on lower shelves and lighter items on higher shelves."

[1155] This enables the system to efficiently manage and quickly search for packages within the logistics center.

[1156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1157] Step 1: Data collection

[1158] The server collects data from camera sensors and RFID sensors installed in the distribution center. The camera sensors capture image data of shelf layout and inventory status, while the RFID sensors read the RFID tag information of each product. This data is sent to the server via the Internet. The input is real-time data from the sensors, and the output is data sent to the server.

[1159] Step 2: Data Preprocessing

[1160] The server preprocesses the collected image data and RFID tag data. This process involves noise removal, data format conversion, and extraction of necessary data. The input is the collected sensor data, and the output is the preprocessed data.

[1161] Step 3: Data analysis

[1162] The server inputs the preprocessed data into a generative AI model and performs analysis to suggest optimal storage locations. For example, TensorFlow is used to consider the weight and size of items and current inventory status to suggest the optimal shelf. The input is the preprocessed data, and the output is storage suggestions generated by the AI.

[1163] Step 4: Notification of storage proposal

[1164] The server notifies the generated storage suggestions to the smartphone or tablet device, which then communicates the specific suggestions to the user through pop-up notifications or highlighting on the UI. The input is the suggestions generated by the AI, and the output is the notified suggestions.

[1165] Step 5: Recording product information

[1166] The user stores the items in the designated locations according to the suggestions and enters the information into the terminal. The terminal sends this information to the server, which records it in the database. The input is the item information sent by the user, and the output is the information recorded in the database.

[1167] Step 6: Search for items

[1168] When a user wants to find a specific item, they input a search query into their device. The server searches the database and obtains the location information of the corresponding item. The obtained information is sent to the device and notified to the user. The input is the user's search query, and the output is the location information of the item.

[1169] These processing steps result in efficient package management and rapid retrieval at the logistics center.

[1170] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1171] The "smart organization support AI system" of the present invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with the management and retrieval of items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. This system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items. Specific embodiments of the system are described below.

[1172] Overall system configuration

[1173] Sensor installation and data collection

[1174] The server collects data in real time from camera and RFID sensors installed in the living space. The camera sensors send captured image data, and the RFID sensors send item ID data to the server. The emotion engine also uses sensors to detect the user's tone of voice, facial expressions, and gestures.

[1175] Data analysis and storage proposals

[1176] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm. The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts its suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[1177] Suggestion notifications and interface

[1178] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[1179] Recording and managing product information

[1180] When a user puts an item away, they input the storage location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in the database.

[1181] Search and Notifications

[1182] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[1183] Specific system operation examples

[1184] Example 1: Organizing your living room

[1185] 1. Sensor installation: Install a camera sensor, RFID sensor, and emotion recognition sensor in the living room.

[1186] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[1187] 3. Data analysis: The server generates storage suggestions using AI algorithms, and an emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[1188] 4. Proposal notification: The terminal notifies the user of this proposal.

[1189] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[1190] 6. Recording information: The server records this information in a database.

[1191] Example 2: Product search

[1192] 1. Search request: The user asks the app, "Where is the remote?"

[1193] 2. Database search: The server searches the database and obtains the location information, such as "The remote control is in the table drawer in the living room."

[1194] 3. Notification: The device notifies the user of this information.

[1195] This system allows for efficient storage method suggestions, item management, and search, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[1196] The processing flow will be explained below.

[1197] Step 1:

[1198] The server collects data in real time from camera sensors, RFID sensors, and emotion recognition sensors installed in the living space. The camera sensors capture image data of the room and furniture layout, while the RFID sensors collect ID data of items. The emotion recognition sensors analyze the user's tone of voice, facial expressions, and gestures to obtain emotional data.

[1199] Step 2:

[1200] The server preprocesses the collected data: for image data, it uses object recognition algorithms to detect items and extract location information; for RFID data, it identifies item IDs and locations; and for emotional data, it analyzes emotional states from voice, facial expressions, and gestures.

[1201] Step 3:

[1202] The server inputs the preprocessed data into an artificial intelligence algorithm to analyze the space usage. As a result of the analysis, it generates an efficient storage method. For example, it makes a specific suggestion such as "It would be good to store textbooks on the shelf next to the sofa in the living room." At this time, an emotion engine adjusts the suggestion based on the user's emotional state. For example, if the user is feeling stressed, it suggests a storage method that is simple and easy to access.

[1203] Step 4:

[1204] The device then notifies the user of the storage suggestions received from the server. The suggestions are displayed as notifications or highlighted on the interface via the smartphone or tablet app. For example, the device may notify the user that "it would be convenient to store textbooks on the shelf on the right side of the living room."

[1205] Step 5:

[1206] The user stores items in the designated location according to the storage method suggested by the user. For example, "store books in the bookshelf in the living room."

[1207] Step 6:

[1208] After storing an item, the user inputs the storage location and type of item into the app. For example, the user might input "Store new books on the bookshelf in the living room."

[1209] Step 7:

[1210] The server records the information entered by the user in a database, which updates the location and stock status of the item in real time.

[1211] Step 8:

[1212] When a user is searching for an item, they text the app, "Where's the remote?", and natural language processing is used to analyze the question.

[1213] Step 9:

[1214] The server searches the database and obtains the location information of the relevant item. For example, the location information is obtained in the form of "The remote control is in the drawer of the table in the living room."

[1215] Step 10:

[1216] The device will notify the user of the acquired location information and can also display a visually easy-to-understand map.

[1217] Step 11:

[1218] The server collects user feedback and updates the AI ​​algorithm and emotion engine, allowing it to learn and improve the accuracy of future suggestions.

[1219] Example 2

[1220] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1221] Currently, there is a lack of efficient methods for storing and managing items in living spaces. Furthermore, it is often time-consuming for users to search for items, making it difficult to effectively utilize living space. Furthermore, existing systems do not provide user-friendly experiences because they do not take into account the user's emotional state. A new system is needed to solve these problems.

[1222] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data from sensors installed in the living space, means for preprocessing the data collected from the sensors and passing it to an artificial intelligence algorithm, means for notifying the user of the storage method generated by the artificial intelligence algorithm, means for the user to input storage locations of items, database means for recording and managing the input storage locations and item information, means for searching for and notifying the location of items based on the database information, and means for analyzing emotional data and making suggestions according to the user's emotional state. This enables efficient storage method suggestions, item management and search, and user-friendly suggestions according to the user's emotional state.

[1223] "Living space" refers to the space and environment in which people live their daily lives.

[1224] A "sensor" refers to a device that detects a physical phenomenon and outputs that information as an electrical signal.

[1225] "Data preprocessing" refers to processes such as trimming, filtering, and decoding of raw data collected from sensors to make it easier to analyze.

[1226] "Artificial intelligence algorithms" refers to mathematical and statistical methods for analyzing data to perform specific tasks, and specifically includes machine learning and deep learning techniques.

[1227] "Storage method" refers to the techniques and procedures for arranging items efficiently and optimally.

[1228] "Notification" refers to the act of notifying a specific device or user of certain information.

[1229] "Input means" refers to the method or device by which a user provides information to a system.

[1230] A "database" refers to a system that systematically records and manages information and enables it to be searched and retrieved as needed.

[1231] "Emotional data" refers to information that indicates a user's emotional state, such as tone of voice, facial expressions, and gestures.

[1232] "User-friendly" refers to a design concept that allows users to operate intuitively and comfortably.

[1233] The "Smart Organizing Support AI System" of this invention is a system that uses sensors installed in living spaces and artificial intelligence to suggest efficient storage methods and assist with managing and searching for items. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to make more user-friendly suggestions. The system aims to provide optimal storage methods that make effective use of limited space and reduce the effort required for users to search for items.

[1234] Overall system configuration

[1235] Sensor installation and data collection

[1236] The server collects data in real time from camera sensors (e.g., Nest Cam) and RFID sensors (e.g., Impinj Speedway R420) installed in the living space. Image data captured by the camera sensors and item ID data acquired by RFID sensors are sent to the server. In addition, sensors used for the emotion engine (e.g., Intrinsic RealSense D435) detect the user's tone of voice, facial expressions, and gestures.

[1237] Data analysis and storage proposals

[1238] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes this data and generates the optimal storage method. For example, it may suggest that "it's best to store textbooks on the shelf next to the sofa in the living room." The emotion engine analyzes the user's emotional data and adjusts the suggestions based on this information. For example, if the user is feeling stressed, it will suggest a storage method that is simple and easy to access.

[1239] Suggestion notifications and interface

[1240] The device notifies the user of storage suggestions generated by the AI ​​algorithm and emotion engine. The suggestions are provided via a smartphone or tablet app (e.g., iOS or Android app) as pop-up notifications or highlighted on the UI. The user then uses these suggestions to store items in the designated locations.

[1241] Recording and managing product information

[1242] When a user puts an item away, they input the location and type of item into the app. For example, they might enter "Put new books on the bookshelf in the living room." This information is sent to the server and recorded in a database (e.g., MySQL or PostgreSQL).

[1243] Search and Notifications

[1244] When a user is looking for something, they can ask the app, "Where's the remote?" The server searches the database to get the location information and sends it to the device, which then notifies the user and, in some cases, displays an easy-to-read map.

[1245] Specific system operation examples

[1246] Example 1: Organizing your living room

[1247] 1. Sensor installation: Install a camera sensor (e.g., Nest Cam), an RFID sensor (e.g., Impinj Speedway R420), and an emotion recognition sensor (e.g., Intel RealSense D435) in the living room.

[1248] 2. Data collection: The server collects furniture layout and object location data, as well as user emotion data, from sensors.

[1249] 3. Data analysis: The server generates storage suggestions using AI algorithms (e.g., TensorFlow), and the emotion engine adjusts the suggestions based on the user's emotions. For example, it might suggest, "It would be convenient to store magazines in the storage box under the table next to the sofa in the living room."

[1250] 4. Proposal notification: The terminal notifies the user of this proposal.

[1251] 5. Input information: The user inputs into the app, "Put the magazines in the box next to the sofa."

[1252] 6. Recording information: The server records this information in a database (e.g. MySQL).

[1253] Example 2: Product search

[1254] 1. Search request: The user asks the app, "Where is the remote?"

[1255] 2. Database search: The server searches the database (e.g., PostgreSQL) and obtains the location information, such as "The remote control is in the drawer of the table in the living room."

[1256] 3. Notification: The device notifies the user of this information.

[1257] Examples of prompt statements

[1258] Below are some examples of prompt sentences to input into the generative AI model.

[1259] "Please suggest the optimal storage method for the living room. Please generate an efficient storage method using furniture layout data and item location data collected by camera sensors and RFID sensors, and user emotion data collected by emotion recognition sensors."

[1260] This system allows for efficient storage method suggestions, item management, and searching, and also makes suggestions based on the user's emotional state, thereby improving the user's quality of life.

[1261] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1262] Step 1: Sensor installation and data collection

[1263] The server initializes the camera sensor (e.g., Nest Cam), RFID sensor (e.g., Impinj Speedway R420), and emotion recognition sensor (e.g., RealSense D435). It collects data from the sensors in real time. As input, it receives image data from the camera sensor, item ID data from the RFID sensor, and facial expression and voice tone data from the emotion recognition sensor. The output is a set of these sensor data.

[1264] Step 2: Data Preprocessing

[1265] The server preprocesses the collected sensor data. It crops or filters the image data from the camera sensor to remove noise, decodes the data from the RFID sensor and converts it into an item ID, and analyzes the data obtained from the emotion recognition sensor to determine the user's emotional state. The inputs are the image data, RFID data, and emotion data collected in step 1. The outputs are the cropped image data, decoded item ID, and analyzed emotion data.

[1266] Step 3: Analysis by AI algorithm

[1267] The server inputs the cropped image data, decoded item IDs, and analyzed emotion data into an artificial intelligence algorithm (e.g., TensorFlow or PyTorch). The AI ​​algorithm analyzes these data and generates an optimal storage method. The input is the preprocessed data output from Step 2. The output is a specific storage suggestion (e.g., "Store your textbooks on the shelf next to the sofa in the living room").

[1268] Step 4: Adjusting suggestions with the emotion engine

[1269] The server adjusts the storage suggestions based on the AI ​​algorithm and the analyzed emotional data. For example, if the user is feeling stressed, it will prepare an easy storage method that makes it easy to access. The input is the storage suggestions generated in step 3 and the emotional data. The output is the storage suggestions adjusted according to the user's emotional state.

[1270] Step 5: Proposal Notification and Interface

[1271] The device receives the adjusted storage suggestions from the server. The suggestions are displayed as pop-up notifications or on the UI via a smartphone or tablet app. The input is notification data containing the adjusted storage suggestions. The output is the specific suggestions presented to the user.

[1272] Step 6: Record and manage product information

[1273] The user stores the item in a designated location and enters the storage location into the app. The device sends this information to the server, which records it in the database. The input is the item storage location data entered by the user into the app. The output is the updated database information.

[1274] Step 7: Search for and notify items

[1275] When a user asks the app, "Where's the remote?", the server searches the database and retrieves the location information of the item. The device notifies the user of this information. The input is the search query for the item. The output is the location information of the found item.

[1276] (Application example 2)

[1277] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1278] Inventory management and product placement in physical stores require a lot of time and effort, and poses a significant workload for store staff. Furthermore, store staff stress and busyness can affect inventory management and product placement, making it difficult to carry out their work efficiently. In order for store staff to work efficiently under these circumstances, they need to be able to grasp the situation in real time and receive accurate instructions. However, conventional systems have not been able to fully achieve this.

[1279] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1280] In this invention, the server includes: means for collecting data from sensors installed in the living space; artificial intelligence means for analyzing the data collected from the sensors and proposing storage methods; means for recognizing the user's emotions and reflecting them in the suggestions; means for the user to input the storage locations of items; database means for recording and managing the input storage locations and item information; means for searching for and notifying the location of items based on the database information; and means for notifying and displaying the suggestions on a device such as smart glasses or a tablet. This allows for optimal inventory management and product placement suggestions to be made in real time, even in physical stores, reducing the workload of store clerks and enabling more efficient work. Furthermore, by providing optimal suggestions based on the emotional state of store clerks, stress and fatigue can be reduced, improving work efficiency.

[1281] "Sensors installed in residential spaces" refers to devices, such as camera sensors and RFID sensors, installed to collect information about items and the environment within a specific space.

[1282] "Means of collecting data from sensors" refers to methods, equipment, and software for collecting data obtained from camera sensors, RFID sensors, etc. in real time or periodically.

[1283] "Artificial intelligence means for analyzing data and suggesting storage solutions" refers to AI algorithms and related software used to process and analyze collected data and generate and suggest optimal storage solutions and layouts.

[1284] "Means of recognizing users' emotions and reflecting them in proposals" refers to methods, equipment, and software that use sensors to obtain emotional data such as users' facial expressions and tone of voice, analyze it, and reflect it in proposal content.

[1285] "Means for inputting the storage location of an item" refers to an interface, device, or software that allows a user to input where an item has been stored into the system.

[1286] "Database means" refers to a database system and related software for recording and managing collected data and entered information.

[1287] "Means for searching for and notifying the location of an item based on database information" refers to a method, device, or software for searching for the location of an item using information recorded in a database and notifying the user of the results.

[1288] "Means for notifying and displaying suggestion content on devices such as smart glasses and tablets" refers to applications or software for displaying suggestions and notification content generated by AI on electronic devices such as smart glasses, tablets, and smartphones.

[1289] The "inventory management and product placement proposal system for brick-and-mortar stores" of the present invention is a system that uses in-store sensors and artificial intelligence to propose efficient inventory management and product placement, thereby reducing the workload of store staff. Specific embodiments are described below.

[1290] Sensor installation and data collection

[1291] The server collects data in real time from camera sensors and RFID sensors installed in the store. The camera sensors capture image data of the shelves, and the RFID sensors read the ID data of each product. The emotion engine also uses sensors to detect the tone of voice and facial expressions of store staff. This data is sent to the server for preprocessing in preparation for the next analysis step.

[1292] Data analysis and proposal generation

[1293] The server preprocesses the collected sensor data and passes it as input parameters to an artificial intelligence algorithm (e.g., TensorFlow). The AI ​​algorithm analyzes this data and generates optimal inventory management and product placement suggestions. For example, it may make a suggestion such as, "Popular products need to be replenished." The emotion engine analyzes the emotional data of the store clerk and adjusts the suggestions based on this information. For example, if the store clerk is feeling stressed, it may make suggestions that are easy to implement.

[1294] Suggestion notifications and interface

[1295] The device (e.g., smart glasses or tablet) generates suggestions based on the AI ​​algorithm and emotion engine and notifies the salesperson. The suggestions are provided via the device's app as pop-up notifications or highlighted in the UI. The salesperson can then use this information to replenish or rearrange products.

[1296] Recording and managing inventory information

[1297] The user (store clerk) inputs product placement and inventory data into the terminal. For example, they input "add new product to shelf." This information is sent to the server and recorded in the database.

[1298] Search and Notifications

[1299] When a user wants to check on a particular product or inventory, they can ask the device, "Where is product X?" The server searches the database to get the location information and sends it to the device, which then reports this information to a store associate and, in some cases, provides an easy-to-read map display of the location.

[1300] By implementing the above, store inventory management and product placement can be carried out efficiently, and suggestions can be made based on the emotional state of store staff, reducing their workload and improving the efficiency of store operations.

[1301] Examples of prompt statements

[1302] "Write a program that generates optimal suggestions to optimize product placement in a store using data collected from camera sensors and RFID sensors, as well as the emotional data of store clerks. In particular, make easy suggestions when store clerks are stressed, and suggest the optimal placement method when they are not."

[1303] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1304] Step 1:

[1305] The server collects data in real time from camera sensors and RFID sensors installed in the store. Specifically, the camera sensors take images of the shelves, and the RFID sensors read the product ID data. This collected data is sent to the server. The inputs are image data and ID data, and the output is saved in an integrated format.

[1306] Step 2:

[1307] The server preprocesses the collected sensor data. Image data is preprocessed using image processing libraries such as OpenCV to remove noise and adjust resolution. Meanwhile, RFID data is checked for duplication and errors and formatted as clean data. The input is the collected data, and the output is preprocessed clean data.

[1308] Step 3:

[1309] The server inputs the preprocessed data into an AI algorithm. For example, TensorFlow is used to detect available shelf space and product counts from image data. This processing generates optimal inventory management and product placement recommendations. The input is the preprocessed data, and the output is the recommendations.

[1310] Step 4:

[1311] The server uses an emotion engine to analyze the clerk's emotional data. It uses an emotion recognition API (e.g., Microsoft Azure Cognitive Services) to analyze the clerk's tone of voice and facial expressions, and reflects the results in the recommendations. The input is emotional data obtained from the sensor, and the output is the emotion analysis result.

[1312] Step 5:

[1313] The server integrates the results of the AI ​​algorithm and emotion engine to generate the final proposal. For example, if the store clerk is busy, it will suggest a replenishment that can be easily implemented, and if not, it will suggest the optimal placement. The input is the results of the AI ​​algorithm and emotion engine, and the integrated proposal is obtained as the output.

[1314] Step 6:

[1315] The device notifies and displays the generated proposal on the smart glasses or tablet. Specifically, a pop-up notification is displayed in the app or a highlight is displayed on the UI. The input is the final proposal, and the proposal is notified to the store clerk as the output.

[1316] Step 7:

[1317] The user (store clerk) replenishes and arranges products according to the suggestions displayed on the terminal. Specific actions include moving or replenishing products to the suggested locations. The input is the suggestions, and the output is the completed replenishment and new arrangement information.

[1318] Step 8:

[1319] The user inputs product placement and inventory data into the device. For example, they input information such as "add a new product to the shelf" into the app. The input is product placement information, and the output is updated database information.

[1320] Step 9:

[1321] The server searches for the location of the item based on the database information and notifies the user. When a user checks a specific product or inventory, the server searches the database to obtain location information and sends the results to the device. The input is a search query, and the output is location information.

[1322] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1323] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1324] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1325] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1326] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1327] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1328] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1329] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1330] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1331] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1332] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1333] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1334] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1335] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1336] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1337] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1338] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1339] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1340] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1341] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1342] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1343] The following is further disclosed regarding the above embodiment.

[1344] (Claim 1)

[1345] a means for collecting data from sensors installed in the living space;

[1346] an artificial intelligence means for analyzing data collected from the sensors and proposing a storage method;

[1347] A means for a user to input the storage location of an item;

[1348] a database means for recording and managing the input storage location and item information;

[1349] means for searching for and notifying the location of an item based on the database information;

[1350] A system including:

[1351] (Claim 2)

[1352] The system of claim 1, wherein the artificial intelligence means includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the content of the suggestions.

[1353] (Claim 3)

[1354] The system of claim 1 , wherein the sensors include a camera sensor and an RFID sensor.

[1355] "Example 1"

[1356] (Claim 1)

[1357] a means for collecting data from sensors installed in the living space;

[1358] means for pre-processing data collected from said sensors;

[1359] artificial intelligence means for analyzing the pre-processed data and suggesting storage methods;

[1360] a means for informing the user of the proposed storage method;

[1361] A means for a user to input the storage location of an item;

[1362] a database means for recording and managing the input storage location and item information;

[1363] means for searching for and notifying the location of an item based on the database information;

[1364] A system including:

[1365] (Claim 2)

[1366] The system of claim 1, wherein the artificial intelligence means includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the content of the suggestions.

[1367] (Claim 3)

[1368] The system of claim 1 , wherein the sensors include a camera sensor and an RFID sensor.

[1369] "Application Example 1"

[1370] (Claim 1)

[1371] a means for collecting data from sensors installed in the living space;

[1372] an artificial intelligence means for analyzing data collected from the sensors and proposing a storage method;

[1373] A means for a user to input the storage location of an item;

[1374] a database means for recording and managing the input storage location and item information;

[1375] means for searching for and notifying the location of an item based on the database information;

[1376] a means for assisting in the storage and retrieval of packages at a logistics center;

[1377] A system including:

[1378] (Claim 2)

[1379] The system of claim 1, wherein the artificial intelligence means includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the content of the suggestions.

[1380] (Claim 3)

[1381] The system of claim 1 , wherein the sensors include a camera sensor and an RFID sensor.

[1382] "Example 2: Combining Emotion Engines"

[1383] (Claim 1)

[1384] a means for collecting data from sensors installed in the living space;

[1385] means for pre-processing data collected from said sensors and passing it to an artificial intelligence algorithm;

[1386] a means for informing a user of the storage method generated by the artificial intelligence algorithm;

[1387] A means for a user to input the storage location of an item;

[1388] a database means for recording and managing the input storage location and item information;

[1389] means for searching for and notifying the location of an item based on the database information;

[1390] A system including a means for analyzing emotional data and making suggestions according to the user's emotional state.

[1391] (Claim 2)

[1392] 2. The system of claim 1, wherein the artificial intelligence algorithm includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the recommendations.

[1393] (Claim 3)

[1394] The system of claim 1 , wherein the sensors include an image sensor and a radio frequency identification sensor.

[1395] "Application example 2 when combining emotion engines"

[1396] (Claim 1)

[1397] a means for collecting data from sensors installed in the living space;

[1398] an artificial intelligence means for analyzing data collected from the sensors and proposing a storage method;

[1399] A means of recognizing user emotions and reflecting them in suggestions;

[1400] A means for a user to input the storage location of an item;

[1401] a database means for recording and managing the input storage location and item information;

[1402] means for searching for and notifying the location of an item based on the database information;

[1403] A means for notifying and displaying the proposed content on a device such as smart glasses or a tablet;

[1404] A system including:

[1405] (Claim 2)

[1406] The system of claim 1, wherein the artificial intelligence means includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the content of the suggestions.

[1407] (Claim 3)

[1408] The system of claim 1 , wherein the sensors include a camera sensor and an RFID sensor. [Explanation of symbols]

[1409] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting data from sensors installed in the living space; an artificial intelligence means for analyzing data collected from the sensors and proposing a storage method; A means for a user to input the storage location of an item; a database means for recording and managing the input storage location and item information; means for searching for and notifying the location of an item based on the database information; A system including:

2. The system according to claim 1 , wherein the artificial intelligence means includes means for learning the user's lifestyle patterns and preferences through deep learning and improving the content of the suggestions.

3. The system of claim 1 , wherein the sensors include a camera sensor and an RFID sensor.

Images