system

Abstract

We provide the system. [Solution] An image acquisition device provides a means for acquiring video footage of the area around the elevator, A method using an information processing device that analyzes the video and determines whether a specific person needs priority use of the elevator, A means of providing guidance to other users via an information display device, based on the judgment result, encouraging them to use stairs or mobility devices, A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In many modern buildings, elevators are often congested, and situations where it is difficult to use elevators frequently occur for people who indispensably need to use elevators due to physical reasons. As a result, problems have arisen that pregnant women, wheelchair users, and other people with physical restrictions cannot appropriately use elevators.

Means for Solving the Problems

[0005] This invention provides a method for supporting the efficient use of elevators through a system using an image acquisition device and an information processing device. This system acquires images of the area around the elevator and analyzes those images to determine whether a particular person requires priority use of the elevator. Based on this determination, it displays guidance recommending other available modes of transportation, thereby enabling those who need it to use the elevator smoothly. Furthermore, it provides guidance to other passengers inside the elevator, prompting them to disembark at the next floor, thereby supporting those who require priority use of the elevator.

[0006] An "image acquisition device" is a device that captures surrounding video footage and acquires it as data.

[0007] "Analyzing video" is the process of processing acquired video data, extracting various features, and making judgments based on them.

[0008] An "information processing device" is a computing device that analyzes acquired data and makes decisions based on specific conditions.

[0009] "Specific individuals" refers to individuals who are deemed to require priority use of elevators.

[0010] An "elevator" is a mechanical lift used to move between different floors within a building.

[0011] "Priority use" refers to a situation where a specific person has the right to use an elevator first.

[0012] An "information display device" is a device that conveys information to users in digital or analog form.

[0013] "Guidance" refers to the act of providing instructions or information to users to encourage them to take action. [Brief explanation of the drawing]

[0014] [Figure 1]It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system that optimizes the use of elevators and assists in ensuring that those who need them can use them preferentially. The system consists of an image acquisition device, an information processing device, and an information display device. The following describes a specific embodiment for carrying out the invention.

[0036] In this system, the terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage to identify specific individuals. A specific algorithm is used for the analysis to identify people who require priority use of the elevator, including pregnant women and wheelchair users.

[0037] Based on this information, the server determines who can use the elevator and displays the result on the terminal via an information display device. Specifically, other users who are determined to be able to use the stairs or escalator will see a message on their screen encouraging them to use that mode of transportation.

[0038] On the other hand, if there is a user on the elevator, the server re-analyzes the video feed from inside the elevator. Based on this analysis, the server will, if necessary, send a message to the terminal informing it of the next floor's disembarkation and indicating the possibility that other passengers may yield the elevator.

[0039] This embodiment provides an environment where people who need it can use the elevator smoothly. The system also collects user feedback and uses that data to improve the accuracy of its analysis algorithms. Users can provide feedback on the appropriateness of the guidance through a dedicated screen, which contributes to system improvement.

[0040] As a concrete example, the device can identify pregnant individuals among those waiting in front of elevators during the morning rush hour and guide them to use the elevator first, while encouraging other users to use the stairs. This ensures that elevators are used efficiently even in crowded situations, allowing those who need them to move quickly.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The terminal captures images of the surroundings through an image acquisition device installed at the entrance of the elevator.

[0044] Step 2:

[0045] The device sends the captured video data to the server. The server then activates an AI model to analyze the received video.

[0046] Step 3:

[0047] The server uses an AI model to analyze people in the video to determine if they are pregnant and whether they are using strollers, wheelchairs, crutches, etc.

[0048] Step 4:

[0049] Based on the analysis results, the server determines whether a particular person needs priority use of the elevator.

[0050] Step 5:

[0051] Based on the determination result, the server sends a message to the terminal encouraging other users to use the stairs or escalator.

[0052] Step 6:

[0053] The terminal displays messages received from the server on an information display device and guides the user to change their travel route.

[0054] Step 7:

[0055] If a user is inside the elevator, a terminal inside the elevator sends video of the current passenger status to the server.

[0056] Step 8:

[0057] The server analyzes the video footage from inside the elevator to determine if there is a user who can give up their seat.

[0058] Step 9:

[0059] If the server determines that it can yield, it sends a message to the terminal inside the elevator prompting passengers to disembark on the next floor.

[0060] Step 10:

[0061] The terminal displays a message on the information display device inside the elevator, informing passengers to disembark.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] The aim is to provide a method that allows necessary users to reach their destinations smoothly in situations where efficient use of elevators is difficult. Furthermore, it aims to improve overall travel efficiency by providing flexible guidance tailored to usage conditions.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for acquiring images using an imaging device that captures the surrounding area; means for utilizing an information processing system that analyzes the images to determine whether a particular individual requires preferential use; and means for collecting user feedback through a dedicated interface and supporting the improvement of the accuracy of the analysis algorithm using that data. This makes it possible to provide an environment in which individuals who need it can use the elevator preferentially, thereby improving overall mobility efficiency.

[0067] "Condition of the surrounding area" refers to the physical environment near the elevator and the movements of people within it, and this information is acquired by the imaging device.

[0068] An "imaging device" refers to a camera or other video capture device used to acquire video data in real time.

[0069] "Analyzing video" refers to processing acquired video data and performing computational operations to identify individuals who meet specific characteristics or conditions. This process utilizes algorithms and machine learning techniques.

[0070] An "information processing system" refers to a system consisting of computing resources and software for analyzing video, evaluating data, and generating judgment results.

[0071] "Specific individuals" refers to people who require priority use of elevators, including, for example, pregnant women or those who require physical assistance.

[0072] A "display device" refers to a monitor or screen used to visually present information, and is used to display guidance messages to users.

[0073] "Collecting opinions through a dedicated interface" refers to obtaining information through a software or hardware interface specifically designed to receive user feedback.

[0074] "Improving the accuracy of analysis algorithms" refers to the process of improving the system's judgment and recognition capabilities by using newly collected data and feedback.

[0075] This invention implements a system that enables the efficient use of elevators. The system mainly consists of terminals, servers, and users.

[0076] Terminal role:

[0077] The terminal is equipped with imaging devices to acquire video footage of the area around and inside the elevator. This includes high-resolution cameras and video capture devices. The terminal uses these devices to acquire video data in real time and transmits it to the server in digital format.

[0078] Server role:

[0079] The server is responsible for analyzing the received video data. The generative AI model used here utilizes image recognition technology and implements algorithms to identify specific individuals who require priority access, such as pregnant women or wheelchair users. Frameworks such as TENSORFLOW® and PyTorch are used in this analysis process. Based on the analysis results, the server determines the user's priority and sends data to display the necessary information on the terminal via an information display device.

[0080] Furthermore, the server collects user feedback and uses it to improve the accuracy of the analysis algorithm. This process involves analyzing the feedback data and generating prompts to update the model. An example of a specific prompt is, "Based on user feedback data, please propose solutions to improve the accuracy of identifying priority elevator users."

[0081] User roles:

[0082] Users act based on information presented by the system. Users with priority access to the elevator are guided to move, while other users are encouraged to use alternative means of transport. Users also provide feedback through a dedicated interface, contributing to system improvements in the process.

[0083] A notable feature of this system is its ability to monitor the situation in real time and provide appropriate instructions to individuals, enabling efficient use of elevators even during peak hours.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The terminal acquires video footage of the area around and inside the elevator using an imaging device. Specifically, the terminal activates a high-resolution camera and captures video in real time. The acquired video data is converted into a digital format and sent to the server as is.

[0087] Step 2:

[0088] The server analyzes the video data received from the terminal. The server uses a generative AI model to perform image recognition processing on the digital video data received as input. Frameworks such as TensorFlow are used for data calculations to identify specific individuals in the video (e.g., pregnant women or wheelchair users). The output of this process is a list of identified priority users.

[0089] Step 3:

[0090] The server generates guidance information for users based on the analysis results. The server evaluates the list of priority users and creates guidance messages to send to terminals via display devices. Specifically, the guidance created includes recommendations for priority users to use elevators and messages encouraging other users to use stairs or escalators.

[0091] Step 4:

[0092] The terminal displays the guidance information received from the server on the display device. Specifically, the terminal displays the message in the appropriate location on the screen, providing the user with visual information. At this time, user profile information is also used to determine which user should view which guidance.

[0093] Step 5:

[0094] Users act based on the displayed instructions. Priority users use the elevator, while other users use the recommended alternative. Furthermore, feedback on the appropriateness of the instructions can be provided through a dedicated interface.

[0095] Step 6:

[0096] The server analyzes the feedback data received from the user. The server receives the feedback data as input and generates prompt statements based on this data to improve the model's accuracy. These prompt statements are used to improve the algorithm in the next iteration, contributing to overall system performance improvement.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In public and commercial facilities, a problem exists where people who require priority access to elevators and escalators cannot receive service quickly and smoothly, resulting in reduced convenience. To address this issue, it is necessary to provide optimal guidance to users and improve the overall operational efficiency of elevators and escalators.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for acquiring surrounding images using an image acquisition device, means for determining whether priority use is necessary using an information processing device, and means for providing guidance on a mobile terminal via an information display device. This makes it possible for people who require priority to quickly use the elevator, provide appropriate navigation guidance to other users, and improve overall travel efficiency.

[0102] A "video acquisition device" is a device that collects visual information from the surroundings, and uses cameras or other devices to visualize the situation in front of and behind the elevator.

[0103] An "information processing device" is a computing device that analyzes acquired video data and determines whether a particular person needs priority use of the elevator.

[0104] An "information display device" is a device that provides guidance information to mobile terminals and smart devices and displays appropriate instructions to users.

[0105] A "smart device" is an electronic device that uses mobile terminals and related technologies to provide information necessary for using elevators.

[0106] A "generative AI model" is an artificial intelligence computational model used to support decision-making in video analysis and to identify specific users.

[0107] This invention is a system for optimizing the use of elevators and effectively assisting users who require priority. The system primarily comprises hardware such as a video acquisition device, an information processing device, and an information display device, and utilizes a generative AI model as software.

[0108] The server uses a video acquisition device to collect images of the area around the elevator. This video data is analyzed by an information processing device, and a generative AI model is used to identify specific users, such as pregnant women or wheelchair users. This allows the system to identify individuals who require priority access and provide guidance information to their mobile devices via smart devices.

[0109] The terminal provides optimal navigation guidance to other users via an information display device. This improves the efficiency of elevator use and enhances convenience. The system has a mechanism to continuously improve the accuracy of guidance by collecting and analyzing user feedback using the information display device.

[0110] As a concrete example, if a pregnant woman tries to use an elevator in a shopping mall, the system can automatically prioritize her and direct others to use the stairs. This guidance is trained to effectively operate a generative AI model through explicitly stated prompts. An example prompt would be, "Please describe in detail the system that uses cameras to capture people waiting for elevators in a shopping mall and identifies and displays those who need priority access."

[0111] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0112] Step 1:

[0113] The server captures surrounding video data using a video acquisition device. In this process, cameras monitor the front and back of the elevator, collecting images in real time. The input is video data from the cameras, and the output is video frames that can be analyzed. The obtained data is sent to an information processing device for further processing.

[0114] Step 2:

[0115] The server receives video frames transmitted by the information processing device and performs data analysis using a generative AI model. The input is the video frames acquired in step 1. The generative AI model performs a process to identify specific individuals (e.g., pregnant women or wheelchair users). The output is the result that individuals requiring priority use have been identified. This identification result is then passed on to the next step within the system.

[0116] Step 3:

[0117] The information processing device displays instructions on the terminal via the information display device based on the identified results. The input is the output of step 2 (the identification result of the priority person). Based on this, the system informs mobile terminals and smart devices that priority users have the right to use the elevator, and encourages other users to use other means of transportation such as stairs. The output is the action displayed as an instruction message.

[0118] Step 4:

[0119] The user navigates based on directions provided by the server. User feedback is then input back into the system and analyzed by an information processing device. The input is user feedback data. This feedback is used to evaluate the validity and convenience of the directions and to improve the accuracy of future directions. The output is the analysis result, which contributes to the continuous improvement of the system.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention is a system that provides priority to elevator users, and by combining it with an emotion engine, it enables guidance that also takes into account the emotional state of the user. This system consists of an image acquisition device, an information processing device, an information display device, and an emotion engine.

[0122] The terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage and activates an AI model to recognize specific individuals. The analysis uses specific algorithms to determine whether the person is pregnant or using a stroller, wheelchair, or crutches.

[0123] The server determines whether a particular person needs priority use of the elevator or escalator. Based on this determination, the server sends and displays a message to other users via an information display device, encouraging them to use the stairs or escalator.

[0124] Furthermore, this system incorporates an emotion engine to recognize the user's emotional state. The server analyzes the user's facial expressions and body movements from the acquired video, and the emotion engine determines the user's stress and anxiety levels. Based on this information, the server can dynamically adjust the content and priority of the guidance it provides.

[0125] For example, if a user is showing signs of high stress, the server can use the emotion engine's assessment to avoid prioritizing the use of stairs or mobility devices, and instead guide the user towards using an elevator more smoothly. This ensures that elevators are available to those who need them, while taking into consideration the user's emotional state.

[0126] As a concrete example, if a terminal identifies a pregnant user exhibiting stress in front of an elevator during peak hours, and the emotion engine determines that the user has a high stress level, the server will guide this user to use a priority elevator and recommend alternative transportation methods to other users. This allows for flexible responses tailored to user preferences even during peak hours, improving overall efficiency.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The terminal uses an image acquisition device installed around the elevator to capture video footage of the surroundings.

[0130] Step 2:

[0131] The terminal sends the captured video data to the server, which then activates an AI model for video analysis.

[0132] Step 3:

[0133] The server analyzes the person in the video to determine if they are pregnant or using a stroller, wheelchair, or crutches.

[0134] Step 4:

[0135] The server recognizes a specific person and evaluates whether that person needs priority use of the elevator.

[0136] Step 5:

[0137] Based on the recognition results, the server sends a message to the terminal encouraging other users to use the stairs or escalator if the user should have priority. The terminal then displays this message on its information display device.

[0138] Step 6:

[0139] The device uses an emotion engine to analyze the user's facial expressions from video and measure their stress and anxiety levels.

[0140] Step 7:

[0141] The server receives the results from the emotion engine and dynamically adjusts the guidance content based on the user's emotional state. For example, if the user is emotionally unstable and stressed, a special guidance message will encourage them to use the elevator.

[0142] Step 8:

[0143] The terminal displays a pre-arranged message from the server on the information display device, informing the user whether to use the elevator or choose an alternative mode of transportation.

[0144] Step 9:

[0145] If the user is inside the elevator, the device will capture video again inside the elevator and send it to the server.

[0146] Step 10:

[0147] The server analyzes the video footage inside the elevator and, if there is a user who should give way, sends a message to the terminal prompting them to get off on the next floor.

[0148] Step 11:

[0149] The terminal displays a message on the information display device inside the elevator, providing passengers with appropriate guidance for disembarking.

[0150] Step 12:

[0151] The server provides a feedback function to collect user opinions and use them to improve the accuracy of the guidance provided.

[0152] (Example 2)

[0153] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0154] Conventional mobility systems have insufficient consideration for users' physical conditions, making it difficult to prioritize specific individuals. Furthermore, they lacked the ability to provide guidance that took users' emotional states into account, resulting in insufficient effective responses to alleviate user stress and anxiety. It is necessary to improve this situation and provide users with more comfortable and efficient means of transportation.

[0155] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0156] In this invention, the server includes means for acquiring video footage of the area around the mobile device using an image acquisition device, means for analyzing the video footage to determine the need for priority use by a specific person, and means for evaluating the user's emotional state using emotion analysis technology and adjusting the guidance content. This optimizes the use of the mobile device based on the user's physical and emotional state, enabling comfortable and efficient guidance.

[0157] An "image acquisition device" is a device that captures video footage of the area around a mobile device and collects that data.

[0158] An "information processing device" is a computer that analyzes acquired video data to determine the priority use necessity and emotional state of a specific person.

[0159] An "information display device" is a display device that provides guidance to other users based on analysis results.

[0160] "Emotional analysis technology" is a technology that analyzes the user's facial expressions and posture from video data to evaluate their emotional state, such as stress and anxiety.

[0161] "Priority use necessity" refers to the criteria used to determine whether a particular person should have priority use of a mobility device.

[0162] "Adjusting the information provided" refers to the process of dynamically changing the information presented according to the user's emotional state and priorities.

[0163] This invention provides a system that enables priority guidance for specific users and considers the emotional state of users when using a mobile device. First, a terminal uses an image acquisition device installed around the mobile device to acquire video in real time. The acquired video data is transmitted to a server via the internet.

[0164] The server activates an AI model for image processing to analyze the received video data. This model uses a generative AI model to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users, etc.). This model includes algorithms such as face recognition and object detection.

[0165] Furthermore, the server uses emotion analysis technology to evaluate the user's emotional state. Specifically, it analyzes the user's facial expressions and movements from the video, and the emotion engine determines the level of stress and anxiety.

[0166] Based on these analysis results, the server dynamically generates and displays messages via an information display device encouraging other users to use stairs or other means of transportation. For example, if the server is concerned about a pregnant woman with a high stress level, it will prioritize guiding her to use transportation devices, while providing other users with a message such as, "Please use the stairs for the sake of the pregnant woman."

[0167] A concrete example of a prompt sentence for the generative AI model and emotion engine in this invention would be, "Please describe how the guidance system works to reduce stress for pregnant users during peak hours." In this way, the system supports the use of the optimal mode of transportation and provides comfortable guidance based on the user's physical and emotional state.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The terminal uses an image acquisition device to capture video footage of the surroundings of the mobile device. This video data is acquired in real time and transmitted to the server. The input is video footage from surrounding cameras, and the output is video data that can be processed.

[0171] Step 2:

[0172] The server activates an AI model using the received video data. The AI ​​model analyzes the input video data to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users). During this process, data processing such as face recognition and object detection is used to output the results of the person identification.

[0173] Step 3:

[0174] The server uses emotion analysis technology to analyze the user's facial and body movements. Using video data obtained in the previous stage as input, it calculates stress and anxiety levels. During this process, the emotion engine evaluates the user's emotional state, and an emotion score is obtained as output.

[0175] Step 4:

[0176] The server generates guidance messages to be displayed on the information display device based on the determined priority and emotional state. The inputs are the person's priority and emotional score, and the output is a dynamically adjusted guidance message. For example, a pregnant woman experiencing high stress will receive the message, "Please use this service with priority."

[0177] Step 5:

[0178] The terminal displays the generated guidance message on an information display device, encouraging other users to use stairs or other means of transportation. The input is the generated message, and the output is the display of visual guidance. This ensures appropriate navigation guidance.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] When using elevators, it is especially important to assign appropriate priorities to users in crowded environments or those experiencing physical or mental strain, and to support their movement safely and efficiently. However, conventional technology is insufficient in providing guidance that takes into account the physical condition and emotional state of users, which can lead to user dissatisfaction and confusion. Therefore, improving user satisfaction and operational efficiency in elevator usage scenarios is a challenge.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for acquiring video footage of the area around the elevator using an image acquisition device, means for analyzing the user's facial expressions and movements from the video footage and evaluating their stress level using an emotion engine, means for dynamically adjusting the guidance content based on the stress level evaluation and prioritizing the use of the elevator, and means for generating information to be displayed on a smart device. This makes it possible to guide users to use the elevator efficiently based on priority while reducing their burden.

[0184] An "image acquisition device" is a device used to capture video footage of the area around an elevator and transmits the acquired video data to an information processing device.

[0185] An "information processing device" is a device that analyzes acquired video footage to determine whether a particular user requires priority use of the elevator.

[0186] An "information display device" is a device that provides visual guidance from an information processing device to other users, encouraging them to use stairs, mobility facilities, and other such equipment.

[0187] An "emotion engine" is software or a system used to analyze a user's facial expressions and movements and evaluate their emotional state.

[0188] A "smart device" is an electronic device that a user can wear, which displays information in real time and guides the user's actions.

[0189] This invention is a system that provides priority to elevator users and guides them while considering their emotional state. First, image acquisition equipment installed around the elevator acquires video footage. This video footage is transmitted to an information processing device, where specific analysis is performed. This analysis uses specific algorithms and an emotion engine to determine whether the user is pregnant or using a stroller or wheelchair.

[0190] Based on this analysis, the information processing device determines whether a particular user needs priority use of the elevator. The server then uses this determination to provide guidance to other users through an information display device. In this process, the information display device shows other users a message encouraging them to use stairs or other mobility equipment.

[0191] In addition, the system analyzes the user's facial expressions and movements from the acquired video footage to assess their emotional state. To perform this assessment, an emotion engine is activated to determine the user's stress level. Based on this stress level, the server adjusts the guidance content in real time.

[0192] As a concrete example, consider a user using a smart device in a shopping mall. If this user is pushing a stroller and the emotion engine detects a high stress level, the server will prompt the user to prioritize using the elevator. Meanwhile, other users will be guided to use the escalator, thus ensuring a smoother overall flow of traffic.

[0193] By using a generative AI model, we aim to improve the accuracy of guidance and enhance user satisfaction. An example of a prompt to the generative AI model would be: "Determine whether the user wearing a smart device is experiencing stress, and if stress levels are high, suggest a method to guide them to the elevator first."

[0194] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0195] Step 1:

[0196] The terminal acquires video footage of the area around the elevator. This video is captured through an image acquisition device and transmitted to an information processing device. The information processing device receives the video data as input.

[0197] Step 2:

[0198] The server analyzes the received video. First, a generative AI model is used to identify a specific person, and then a specific algorithm is used to determine whether that person is pregnant or using a stroller or wheelchair. The input is video data, and the output generates identification information and priority ratings for the specific person.

[0199] Step 3:

[0200] The emotion engine is activated based on the identification information acquired by the server. It analyzes facial expressions and body movements obtained from the video to determine the user's stress level. The input is again video data, and the output is data related to the emotional state.

[0201] Step 4:

[0202] The server uses the determined stress level and priority information to generate guidance messages for information display devices. The input is stress level and priority information, and the output is a guidance message that will be presented to other users.

[0203] Step 5:

[0204] The server generates guidance messages and sends them to the smart device, encouraging the user to use stairs or mobility facilities. At this stage, the generated messages are displayed to guide the user's actions. The input is the guidance message, and the output is the display on the smart device.

[0205] This series of processes streamlines the overall flow and allows for efficient and considerate guidance to be provided to users.

[0206] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0207] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0208] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0209] [Second Embodiment]

[0210] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0211] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0212] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0213] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0214] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0215] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0216] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0217] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0218] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0219] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0220] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0221] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0222] This invention is a system that optimizes the use of elevators and assists in ensuring that those who need them can use them preferentially. The system consists of an image acquisition device, an information processing device, and an information display device. The following describes a specific embodiment for carrying out the invention.

[0223] In this system, the terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage to identify specific individuals. A specific algorithm is used for the analysis to identify people who require priority use of the elevator, including pregnant women and wheelchair users.

[0224] Based on this information, the server determines who can use the elevator and displays the result on the terminal via an information display device. Specifically, other users who are determined to be able to use the stairs or escalator will see a message on their screen encouraging them to use that mode of transportation.

[0225] On the other hand, if there is a user on the elevator, the server re-analyzes the video feed from inside the elevator. Based on this analysis, the server will, if necessary, send a message to the terminal informing it of the next floor's disembarkation and indicating the possibility that other passengers may yield the elevator.

[0226] This embodiment provides an environment where people who need it can use the elevator smoothly. The system also collects user feedback and uses that data to improve the accuracy of its analysis algorithms. Users can provide feedback on the appropriateness of the guidance through a dedicated screen, which contributes to system improvement.

[0227] As a concrete example, the device can identify pregnant individuals among those waiting in front of elevators during the morning rush hour and guide them to use the elevator first, while encouraging other users to use the stairs. This ensures that elevators are used efficiently even in crowded situations, allowing those who need them to move quickly.

[0228] The following describes the processing flow.

[0229] Step 1:

[0230] The terminal captures images of the surroundings through an image acquisition device installed at the entrance of the elevator.

[0231] Step 2:

[0232] The device sends the captured video data to the server. The server then activates an AI model to analyze the received video.

[0233] Step 3:

[0234] The server uses an AI model to analyze people in the video to determine if they are pregnant and whether they are using strollers, wheelchairs, crutches, etc.

[0235] Step 4:

[0236] Based on the analysis results, the server determines whether a particular person needs priority use of the elevator.

[0237] Step 5:

[0238] Based on the determination result, the server sends a message to the terminal encouraging other users to use the stairs or escalator.

[0239] Step 6:

[0240] The terminal displays messages received from the server on an information display device and guides the user to change their travel route.

[0241] Step 7:

[0242] If a user is inside the elevator, a terminal inside the elevator sends video of the current passenger status to the server.

[0243] Step 8:

[0244] The server analyzes the video footage from inside the elevator to determine if there is a user who can give up their seat.

[0245] Step 9:

[0246] If the server determines that it can yield, it sends a message to the terminal inside the elevator prompting passengers to disembark on the next floor.

[0247] Step 10:

[0248] The terminal displays a message on the information display device inside the elevator, informing passengers to disembark.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] The aim is to provide a method that allows necessary users to reach their destinations smoothly in situations where efficient use of elevators is difficult. Furthermore, it aims to improve overall travel efficiency by providing flexible guidance tailored to usage conditions.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes means for acquiring images using an imaging device that captures the surrounding area; means for utilizing an information processing system that analyzes the images to determine whether a particular individual requires preferential use; and means for collecting user feedback through a dedicated interface and supporting the improvement of the accuracy of the analysis algorithm using that data. This makes it possible to provide an environment in which individuals who need it can use the elevator preferentially, thereby improving overall mobility efficiency.

[0254] "Condition of the surrounding area" refers to the physical environment near the elevator and the movements of people within it, and this information is acquired by the imaging device.

[0255] An "imaging device" refers to a camera or other video capture device used to acquire video data in real time.

[0256] "Analyzing video" refers to processing acquired video data and performing computational operations to identify individuals who meet specific characteristics or conditions. This process utilizes algorithms and machine learning techniques.

[0257] An "information processing system" refers to a system consisting of computing resources and software for analyzing video, evaluating data, and generating judgment results.

[0258] "Specific individuals" refers to people who require priority use of elevators, including, for example, pregnant women or those who require physical assistance.

[0259] A "display device" refers to a monitor or screen used to visually present information, and is used to display guidance messages to users.

[0260] "Collecting opinions through a dedicated interface" refers to obtaining information through a software or hardware interface specifically designed to receive user feedback.

[0261] "Improving the accuracy of analysis algorithms" refers to the process of improving the system's judgment and recognition capabilities by using newly collected data and feedback.

[0262] This invention implements a system that enables the efficient use of elevators. The system mainly consists of terminals, servers, and users.

[0263] Terminal role:

[0264] The terminal is equipped with imaging devices to acquire video footage of the area around and inside the elevator. This includes high-resolution cameras and video capture devices. The terminal uses these devices to acquire video data in real time and transmits it to the server in digital format.

[0265] Server role:

[0266] The server is responsible for analyzing the received video data. The generative AI model used here utilizes image recognition technology and implements algorithms to identify specific individuals who require priority access, such as pregnant women or wheelchair users. Frameworks such as TensorFlow and PyTorch are used in this analysis process. Based on the analysis results, the server determines the user's priority and sends data to the terminal to display the necessary information via an information display device.

[0267] Furthermore, the server collects user feedback and uses it to improve the accuracy of the analysis algorithm. This process involves analyzing the feedback data and generating prompts to update the model. An example of a specific prompt is, "Based on user feedback data, please propose solutions to improve the accuracy of identifying priority elevator users."

[0268] User roles:

[0269] Users act based on information presented by the system. Users with priority access to the elevator are guided to move, while other users are encouraged to use alternative means of transport. Users also provide feedback through a dedicated interface, contributing to system improvements in the process.

[0270] A notable feature of this system is its ability to monitor the situation in real time and provide appropriate instructions to individuals, enabling efficient use of elevators even during peak hours.

[0271] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0272] Step 1:

[0273] The terminal acquires video footage of the area around and inside the elevator using an imaging device. Specifically, the terminal activates a high-resolution camera and captures video in real time. The acquired video data is converted into a digital format and sent to the server as is.

[0274] Step 2:

[0275] The server analyzes the video data received from the terminal. The server uses a generative AI model to perform image recognition processing on the digital video data received as input. Frameworks such as TensorFlow are used for data calculations to identify specific individuals in the video (e.g., pregnant women or wheelchair users). The output of this process is a list of identified priority users.

[0276] Step 3:

[0277] The server generates guidance information for users based on the analysis results. The server evaluates the list of priority users and creates guidance messages to send to terminals via display devices. Specifically, the guidance created includes recommendations for priority users to use elevators and messages encouraging other users to use stairs or escalators.

[0278] Step 4:

[0279] The terminal displays the guidance information received from the server on the display device. Specifically, the terminal displays the message in the appropriate location on the screen, providing the user with visual information. At this time, user profile information is also used to determine which user should view which guidance.

[0280] Step 5:

[0281] Users act based on the displayed instructions. Priority users use the elevator, while other users use the recommended alternative. Furthermore, feedback on the appropriateness of the instructions can be provided through a dedicated interface.

[0282] Step 6:

[0283] The server analyzes the feedback data received from the user. The server receives the feedback data as input and generates a prompt sentence for improving the accuracy of the model based on this data. This prompt sentence is utilized for the next algorithm improvement and contributes to the performance improvement of the entire system.

[0284] (Application Example 1)

[0285] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0286] In public facilities and commercial facilities, there is a problem of reduced convenience due to the fact that people who need priority in using elevators cannot receive services quickly and smoothly. To address this issue, it is necessary to provide optimal guidance to users and improve the overall operating efficiency of elevators.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0288] In this invention, the server includes means for acquiring surrounding images by an image acquisition device, means for determining whether priority use is required by an information processing device, and means for providing guidance on a mobile terminal via an information display device. Thereby, people who need priority can quickly use the elevator, appropriate movement guidance can be provided to other users, and the overall movement efficiency can be improved.

[0289] The "image acquisition device" is a device for collecting surrounding visual information, and is a device that visualizes the situation before and after the elevator using a camera or the like.

[0290] An "information processing device" is a computing device that analyzes acquired video data and determines whether a particular person needs priority use of the elevator.

[0291] An "information display device" is a device that provides guidance information to mobile terminals and smart devices and displays appropriate instructions to users.

[0292] A "smart device" is an electronic device that uses mobile terminals and related technologies to provide information necessary for using elevators.

[0293] A "generative AI model" is an artificial intelligence computational model used to support decision-making in video analysis and to identify specific users.

[0294] This invention is a system for optimizing the use of elevators and effectively assisting users who require priority. The system primarily comprises hardware such as a video acquisition device, an information processing device, and an information display device, and utilizes a generative AI model as software.

[0295] The server uses a video acquisition device to collect images of the area around the elevator. This video data is analyzed by an information processing device, and a generative AI model is used to identify specific users, such as pregnant women or wheelchair users. This allows the system to identify individuals who require priority access and provide guidance information to their mobile devices via smart devices.

[0296] The terminal provides optimal navigation guidance to other users via an information display device. This improves the efficiency of elevator use and enhances convenience. The system has a mechanism to continuously improve the accuracy of guidance by collecting and analyzing user feedback using the information display device.

[0297] As a concrete example, if a pregnant woman tries to use an elevator in a shopping mall, the system can automatically prioritize her and direct others to use the stairs. This guidance is trained to effectively operate a generative AI model through explicitly stated prompts. An example prompt would be, "Please describe in detail the system that uses cameras to capture people waiting for elevators in a shopping mall and identifies and displays those who need priority access."

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The server captures surrounding video data using a video acquisition device. In this process, cameras monitor the front and back of the elevator, collecting images in real time. The input is video data from the cameras, and the output is video frames that can be analyzed. The obtained data is sent to an information processing device for further processing.

[0301] Step 2:

[0302] The server receives video frames transmitted by the information processing device and performs data analysis using a generative AI model. The input is the video frames acquired in step 1. The generative AI model performs a process to identify specific individuals (e.g., pregnant women or wheelchair users). The output is the result that individuals requiring priority use have been identified. This identification result is then passed on to the next step within the system.

[0303] Step 3:

[0304] The information processing device displays guidance to the terminal through the information display device based on the identified result. The input is the output of Step 2 (the identification result of the priority person). Based on this, the system guides the mobile terminal or smart device that the priority user has the right to use the elevator, and prompts other users to use other means of movement such as stairs. The output is an operation displayed as a guidance message.

[0305] Step 4:

[0306] The user moves based on the guidance provided by the server. The user's feedback is input into the system again and analyzed by the information processing device. The input is the user's feedback data. This feedback is used to evaluate the validity and convenience of the guidance and improve the guidance accuracy for subsequent times. The output is the analysis result, which is an operation contributing to the continuous improvement of the system.

[0307] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion specific model 59 and perform specific processing using the user's emotion.

[0308] This invention is a system that provides priorities to elevator users, and by further combining an emotion engine, it enables guidance that also takes into account the user's emotional state. This system is composed of an image acquisition device, an information processing device, an information display device, and an emotion engine.

[0309] First, the terminal acquires the video around the elevator. The acquired video is transmitted to the server, and the server analyzes this video and activates an AI model for recognizing specific people. For the analysis, a specific algorithm for determining whether a person is pregnant or using a baby stroller, wheelchair, or cane is used.

[0310] The server determines whether a particular person needs priority use of the elevator or escalator. Based on this determination, the server sends and displays a message to other users via an information display device, encouraging them to use the stairs or escalator.

[0311] Furthermore, this system incorporates an emotion engine to recognize the user's emotional state. The server analyzes the user's facial expressions and body movements from the acquired video, and the emotion engine determines the user's stress and anxiety levels. Based on this information, the server can dynamically adjust the content and priority of the guidance it provides.

[0312] For example, if a user is showing signs of high stress, the server can use the emotion engine's assessment to avoid prioritizing the use of stairs or mobility devices, and instead guide the user towards using an elevator more smoothly. This ensures that elevators are available to those who need them, while taking into consideration the user's emotional state.

[0313] As a concrete example, if a terminal identifies a pregnant user exhibiting stress in front of an elevator during peak hours, and the emotion engine determines that the user has a high stress level, the server will guide this user to use a priority elevator and recommend alternative transportation methods to other users. This allows for flexible responses tailored to user preferences even during peak hours, improving overall efficiency.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] The terminal uses an image acquisition device installed around the elevator to capture video footage of the surroundings.

[0317] Step 2:

[0318] The terminal sends the captured video data to the server, which then activates an AI model for video analysis.

[0319] Step 3:

[0320] The server analyzes the person in the video to determine if they are pregnant or using a stroller, wheelchair, or crutches.

[0321] Step 4:

[0322] The server recognizes a specific person and evaluates whether that person needs priority use of the elevator.

[0323] Step 5:

[0324] Based on the recognition results, the server sends a message to the terminal encouraging other users to use the stairs or escalator if the user should have priority. The terminal then displays this message on its information display device.

[0325] Step 6:

[0326] The device uses an emotion engine to analyze the user's facial expressions from video and measure their stress and anxiety levels.

[0327] Step 7:

[0328] The server receives the results from the emotion engine and dynamically adjusts the guidance content based on the user's emotional state. For example, if the user is emotionally unstable and stressed, a special guidance message will encourage them to use the elevator.

[0329] Step 8:

[0330] The terminal displays a pre-arranged message from the server on the information display device, informing the user whether to use the elevator or choose an alternative mode of transportation.

[0331] Step 9:

[0332] If the user is inside the elevator, the device will capture video again inside the elevator and send it to the server.

[0333] Step 10:

[0334] The server analyzes the video footage inside the elevator and, if there is a user who should give way, sends a message to the terminal prompting them to get off on the next floor.

[0335] Step 11:

[0336] The terminal displays a message on the information display device inside the elevator, providing passengers with appropriate guidance for disembarking.

[0337] Step 12:

[0338] The server provides a feedback function to collect user opinions and use them to improve the accuracy of the guidance provided.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] Conventional mobility systems have insufficient consideration for users' physical conditions, making it difficult to prioritize specific individuals. Furthermore, they lacked the ability to provide guidance that took users' emotional states into account, resulting in insufficient effective responses to alleviate user stress and anxiety. It is necessary to improve this situation and provide users with more comfortable and efficient means of transportation.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for acquiring video footage of the area around the mobile device using an image acquisition device, means for analyzing the video footage to determine the need for priority use by a specific person, and means for evaluating the user's emotional state using emotion analysis technology and adjusting the guidance content. This optimizes the use of the mobile device based on the user's physical and emotional state, enabling comfortable and efficient guidance.

[0344] An "image acquisition device" is a device that captures video footage of the area around a mobile device and collects that data.

[0345] An "information processing device" is a computer that analyzes acquired video data to determine the priority use necessity and emotional state of a specific person.

[0346] An "information display device" is a display device that provides guidance to other users based on analysis results.

[0347] "Emotional analysis technology" is a technology that analyzes the user's facial expressions and posture from video data to evaluate their emotional state, such as stress and anxiety.

[0348] "Priority use necessity" refers to the criteria used to determine whether a particular person should have priority use of a mobility device.

[0349] "Adjusting the information provided" refers to the process of dynamically changing the information presented according to the user's emotional state and priorities.

[0350] This invention provides a system that enables priority guidance for specific users and considers the emotional state of users when using a mobile device. First, a terminal uses an image acquisition device installed around the mobile device to acquire video in real time. The acquired video data is transmitted to a server via the internet.

[0351] The server activates an AI model for image processing to analyze the received video data. This model uses a generative AI model to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users, etc.). This model includes algorithms such as face recognition and object detection.

[0352] Furthermore, the server uses emotion analysis technology to evaluate the user's emotional state. Specifically, it analyzes the user's facial expressions and movements from the video, and the emotion engine determines the level of stress and anxiety.

[0353] Based on these analysis results, the server dynamically generates and displays messages via an information display device encouraging other users to use stairs or other means of transportation. For example, if the server is concerned about a pregnant woman with a high stress level, it will prioritize guiding her to use transportation devices, while providing other users with a message such as, "Please use the stairs for the sake of the pregnant woman."

[0354] A concrete example of a prompt sentence for the generative AI model and emotion engine in this invention would be, "Please describe how the guidance system works to reduce stress for pregnant users during peak hours." In this way, the system supports the use of the optimal mode of transportation and provides comfortable guidance based on the user's physical and emotional state.

[0355] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0356] Step 1:

[0357] The terminal uses an image acquisition device to capture video footage of the surroundings of the mobile device. This video data is acquired in real time and transmitted to the server. The input is video footage from surrounding cameras, and the output is video data that can be processed.

[0358] Step 2:

[0359] The server activates an AI model using the received video data. The AI ​​model analyzes the input video data to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users). During this process, data processing such as face recognition and object detection is used to output the results of the person identification.

[0360] Step 3:

[0361] The server uses emotion analysis technology to analyze the user's facial and body movements. Using video data obtained in the previous stage as input, it calculates stress and anxiety levels. During this process, the emotion engine evaluates the user's emotional state, and an emotion score is obtained as output.

[0362] Step 4:

[0363] The server generates guidance messages to be displayed on the information display device based on the determined priority and emotional state. The inputs are the person's priority and emotional score, and the output is a dynamically adjusted guidance message. For example, a pregnant woman experiencing high stress will receive the message, "Please use this service with priority."

[0364] Step 5:

[0365] The terminal displays the generated guidance message on an information display device, encouraging other users to use stairs or other means of transportation. The input is the generated message, and the output is the display of visual guidance. This ensures appropriate navigation guidance.

[0366] (Application Example 2)

[0367] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0368] When using elevators, it is especially important to assign appropriate priorities to users in crowded environments or those experiencing physical or mental strain, and to support their movement safely and efficiently. However, conventional technology is insufficient in providing guidance that takes into account the physical condition and emotional state of users, which can lead to user dissatisfaction and confusion. Therefore, improving user satisfaction and operational efficiency in elevator usage scenarios is a challenge.

[0369] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0370] In this invention, the server includes means for acquiring video footage of the area around the elevator using an image acquisition device, means for analyzing the user's facial expressions and movements from the video footage and evaluating their stress level using an emotion engine, means for dynamically adjusting the guidance content based on the stress level evaluation and prioritizing the use of the elevator, and means for generating information to be displayed on a smart device. This makes it possible to guide users to use the elevator efficiently based on priority while reducing their burden.

[0371] An "image acquisition device" is a device used to capture video footage of the area around an elevator and transmits the acquired video data to an information processing device.

[0372] An "information processing device" is a device that analyzes acquired video footage to determine whether a particular user requires priority use of the elevator.

[0373] An "information display device" is a device that provides visual guidance from an information processing device to other users, encouraging them to use stairs, mobility facilities, and other such equipment.

[0374] An "emotion engine" is software or a system used to analyze a user's facial expressions and movements and evaluate their emotional state.

[0375] A "smart device" is an electronic device that a user can wear, which displays information in real time and guides the user's actions.

[0376] This invention is a system that provides priority to elevator users and guides them while considering their emotional state. First, image acquisition equipment installed around the elevator acquires video footage. This video footage is transmitted to an information processing device, where specific analysis is performed. This analysis uses specific algorithms and an emotion engine to determine whether the user is pregnant or using a stroller or wheelchair.

[0377] Based on this analysis, the information processing device determines whether a particular user needs priority use of the elevator. The server then uses this determination to provide guidance to other users through an information display device. In this process, the information display device shows other users a message encouraging them to use stairs or other mobility equipment.

[0378] In addition, the system analyzes the user's facial expressions and movements from the acquired video footage to assess their emotional state. To perform this assessment, an emotion engine is activated to determine the user's stress level. Based on this stress level, the server adjusts the guidance content in real time.

[0379] As a concrete example, consider a user using a smart device in a shopping mall. If this user is pushing a stroller and the emotion engine detects a high stress level, the server will prompt the user to prioritize using the elevator. Meanwhile, other users will be guided to use the escalator, thus ensuring a smoother overall flow of traffic.

[0380] By using a generative AI model, we aim to improve the accuracy of guidance and enhance user satisfaction. An example of a prompt to the generative AI model would be: "Determine whether the user wearing a smart device is experiencing stress, and if stress levels are high, suggest a method to guide them to the elevator first."

[0381] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0382] Step 1:

[0383] The terminal acquires video footage of the area around the elevator. This video is captured through an image acquisition device and transmitted to an information processing device. The information processing device receives the video data as input.

[0384] Step 2:

[0385] The server analyzes the received video. First, a generative AI model is used to identify a specific person, and then a specific algorithm is used to determine whether that person is pregnant or using a stroller or wheelchair. The input is video data, and the output generates identification information and priority ratings for the specific person.

[0386] Step 3:

[0387] The emotion engine is activated based on the identification information acquired by the server. It analyzes facial expressions and body movements obtained from the video to determine the user's stress level. The input is again video data, and the output is data related to the emotional state.

[0388] Step 4:

[0389] The server uses the determined stress level and priority information to generate guidance messages for information display devices. The input is stress level and priority information, and the output is a guidance message that will be presented to other users.

[0390] Step 5:

[0391] The server generates guidance messages and sends them to the smart device, encouraging the user to use stairs or mobility facilities. At this stage, the generated messages are displayed to guide the user's actions. The input is the guidance message, and the output is the display on the smart device.

[0392] This series of processes streamlines the overall flow and allows for efficient and considerate guidance to be provided to users.

[0393] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0394] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0395] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0396] [Third Embodiment]

[0397] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0398] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0399] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0400] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0401] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0402] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0403] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0404] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0405] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0406] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0407] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0408] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0409] This invention is a system that optimizes the use of elevators and assists in ensuring that those who need them can use them preferentially. The system consists of an image acquisition device, an information processing device, and an information display device. The following describes a specific embodiment for carrying out the invention.

[0410] In this system, the terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage to identify specific individuals. A specific algorithm is used for the analysis to identify people who require priority use of the elevator, including pregnant women and wheelchair users.

[0411] Based on this information, the server determines who can use the elevator and displays the result on the terminal via an information display device. Specifically, other users who are determined to be able to use the stairs or escalator will see a message on their screen encouraging them to use that mode of transportation.

[0412] On the other hand, if there is a user on the elevator, the server re-analyzes the video feed from inside the elevator. Based on this analysis, the server will, if necessary, send a message to the terminal informing it of the next floor's disembarkation and indicating the possibility that other passengers may yield the elevator.

[0413] This embodiment provides an environment where people who need it can use the elevator smoothly. The system also collects user feedback and uses that data to improve the accuracy of its analysis algorithms. Users can provide feedback on the appropriateness of the guidance through a dedicated screen, which contributes to system improvement.

[0414] As a concrete example, the device can identify pregnant individuals among those waiting in front of elevators during the morning rush hour and guide them to use the elevator first, while encouraging other users to use the stairs. This ensures that elevators are used efficiently even in crowded situations, allowing those who need them to move quickly.

[0415] The following describes the processing flow.

[0416] Step 1:

[0417] The terminal captures images of the surroundings through an image acquisition device installed at the entrance of the elevator.

[0418] Step 2:

[0419] The device sends the captured video data to the server. The server then activates an AI model to analyze the received video.

[0420] Step 3:

[0421] The server uses an AI model to analyze people in the video to determine if they are pregnant and whether they are using strollers, wheelchairs, crutches, etc.

[0422] Step 4:

[0423] Based on the analysis results, the server determines whether a particular person needs priority use of the elevator.

[0424] Step 5:

[0425] Based on the determination result, the server sends a message to the terminal encouraging other users to use the stairs or escalator.

[0426] Step 6:

[0427] The terminal displays messages received from the server on an information display device and guides the user to change their travel route.

[0428] Step 7:

[0429] If a user is inside the elevator, a terminal inside the elevator sends video of the current passenger status to the server.

[0430] Step 8:

[0431] The server analyzes the video footage from inside the elevator to determine if there is a user who can give up their seat.

[0432] Step 9:

[0433] If the server determines that it can yield, it sends a message to the terminal inside the elevator prompting passengers to disembark on the next floor.

[0434] Step 10:

[0435] The terminal displays a message on the information display device inside the elevator, informing passengers to disembark.

[0436] (Example 1)

[0437] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0438] The aim is to provide a method that allows necessary users to reach their destinations smoothly in situations where efficient use of elevators is difficult. Furthermore, it aims to improve overall travel efficiency by providing flexible guidance tailored to usage conditions.

[0439] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0440] In this invention, the server includes means for acquiring images using an imaging device that captures the surrounding area; means for utilizing an information processing system that analyzes the images to determine whether a particular individual requires preferential use; and means for collecting user feedback through a dedicated interface and supporting the improvement of the accuracy of the analysis algorithm using that data. This makes it possible to provide an environment in which individuals who need it can use the elevator preferentially, thereby improving overall mobility efficiency.

[0441] "Condition of the surrounding area" refers to the physical environment near the elevator and the movements of people within it, and this information is acquired by the imaging device.

[0442] An "imaging device" refers to a camera or other video capture device used to acquire video data in real time.

[0443] "Analyzing video" refers to processing acquired video data and performing computational operations to identify individuals who meet specific characteristics or conditions. This process utilizes algorithms and machine learning techniques.

[0444] An "information processing system" refers to a system consisting of computing resources and software for analyzing video, evaluating data, and generating judgment results.

[0445] "Specific individuals" refers to people who require priority use of elevators, including, for example, pregnant women or those who require physical assistance.

[0446] A "display device" refers to a monitor or screen used to visually present information, and is used to display guidance messages to users.

[0447] "Collecting opinions through a dedicated interface" refers to obtaining information through a software or hardware interface specifically designed to receive user feedback.

[0448] "Improving the accuracy of analysis algorithms" refers to the process of improving the system's judgment and recognition capabilities by using newly collected data and feedback.

[0449] This invention implements a system that enables the efficient use of elevators. The system mainly consists of terminals, servers, and users.

[0450] Terminal role:

[0451] The terminal is equipped with imaging devices to acquire video footage of the area around and inside the elevator. This includes high-resolution cameras and video capture devices. The terminal uses these devices to acquire video data in real time and transmits it to the server in digital format.

[0452] Server role:

[0453] The server is responsible for analyzing the received video data. The generative AI model used here utilizes image recognition technology and implements algorithms to identify specific individuals who require priority access, such as pregnant women or wheelchair users. Frameworks such as TensorFlow and PyTorch are used in this analysis process. Based on the analysis results, the server determines the user's priority and sends data to the terminal to display the necessary information via an information display device.

[0454] Furthermore, the server collects user feedback and uses it to improve the accuracy of the analysis algorithm. This process involves analyzing the feedback data and generating prompts to update the model. An example of a specific prompt is, "Based on user feedback data, please propose solutions to improve the accuracy of identifying priority elevator users."

[0455] User roles:

[0456] Users act based on information presented by the system. Users with priority access to the elevator are guided to move, while other users are encouraged to use alternative means of transport. Users also provide feedback through a dedicated interface, contributing to system improvements in the process.

[0457] A notable feature of this system is its ability to monitor the situation in real time and provide appropriate instructions to individuals, enabling efficient use of elevators even during peak hours.

[0458] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0459] Step 1:

[0460] The terminal acquires video footage of the area around and inside the elevator using an imaging device. Specifically, the terminal activates a high-resolution camera and captures video in real time. The acquired video data is converted into a digital format and sent to the server as is.

[0461] Step 2:

[0462] The server analyzes the video data received from the terminal. The server uses a generative AI model to perform image recognition processing on the digital video data received as input. Frameworks such as TensorFlow are used for data calculations to identify specific individuals in the video (e.g., pregnant women or wheelchair users). The output of this process is a list of identified priority users.

[0463] Step 3:

[0464] The server generates guidance information for users based on the analysis results. The server evaluates the list of priority users and creates guidance messages to send to terminals via display devices. Specifically, the guidance created includes recommendations for priority users to use elevators and messages encouraging other users to use stairs or escalators.

[0465] Step 4:

[0466] The terminal displays the guidance information received from the server on the display device. Specifically, the terminal displays the message in the appropriate location on the screen, providing the user with visual information. At this time, user profile information is also used to determine which user should view which guidance.

[0467] Step 5:

[0468] Users act based on the displayed instructions. Priority users use the elevator, while other users use the recommended alternative. Furthermore, feedback on the appropriateness of the instructions can be provided through a dedicated interface.

[0469] Step 6:

[0470] The server analyzes the feedback data received from the user. The server receives the feedback data as input and generates prompt statements based on this data to improve the model's accuracy. These prompt statements are used to improve the algorithm in the next iteration, contributing to overall system performance improvement.

[0471] (Application Example 1)

[0472] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] In public and commercial facilities, a problem exists where people who require priority access to elevators and escalators cannot receive service quickly and smoothly, resulting in reduced convenience. To address this issue, it is necessary to provide optimal guidance to users and improve the overall operational efficiency of elevators and escalators.

[0474] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0475] In this invention, the server includes means for acquiring surrounding images using an image acquisition device, means for determining whether priority use is necessary using an information processing device, and means for providing guidance on a mobile terminal via an information display device. This makes it possible for people who require priority to quickly use the elevator, provide appropriate navigation guidance to other users, and improve overall travel efficiency.

[0476] A "video acquisition device" is a device that collects visual information from the surroundings, and uses cameras or other devices to visualize the situation in front of and behind the elevator.

[0477] An "information processing device" is a computing device that analyzes acquired video data and determines whether a particular person needs priority use of the elevator.

[0478] An "information display device" is a device that provides guidance information to mobile terminals and smart devices and displays appropriate instructions to users.

[0479] A "smart device" is an electronic device that uses mobile terminals and related technologies to provide information necessary for using elevators.

[0480] A "generative AI model" is an artificial intelligence computational model used to support decision-making in video analysis and to identify specific users.

[0481] This invention is a system for optimizing the use of elevators and effectively assisting users who require priority. The system primarily comprises hardware such as a video acquisition device, an information processing device, and an information display device, and utilizes a generative AI model as software.

[0482] The server uses a video acquisition device to collect images of the area around the elevator. This video data is analyzed by an information processing device, and a generative AI model is used to identify specific users, such as pregnant women or wheelchair users. This allows the system to identify individuals who require priority access and provide guidance information to their mobile devices via smart devices.

[0483] The terminal provides optimal navigation guidance to other users via an information display device. This improves the efficiency of elevator use and enhances convenience. The system has a mechanism to continuously improve the accuracy of guidance by collecting and analyzing user feedback using the information display device.

[0484] As a concrete example, if a pregnant woman tries to use an elevator in a shopping mall, the system can automatically prioritize her and direct others to use the stairs. This guidance is trained to effectively operate a generative AI model through explicitly stated prompts. An example prompt would be, "Please describe in detail the system that uses cameras to capture people waiting for elevators in a shopping mall and identifies and displays those who need priority access."

[0485] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0486] Step 1:

[0487] The server captures surrounding video data using a video acquisition device. In this process, cameras monitor the front and back of the elevator, collecting images in real time. The input is video data from the cameras, and the output is video frames that can be analyzed. The obtained data is sent to an information processing device for further processing.

[0488] Step 2:

[0489] The server receives video frames transmitted by the information processing device and performs data analysis using a generative AI model. The input is the video frames acquired in step 1. The generative AI model performs a process to identify specific individuals (e.g., pregnant women or wheelchair users). The output is the result that individuals requiring priority use have been identified. This identification result is then passed on to the next step within the system.

[0490] Step 3:

[0491] The information processing device displays instructions on the terminal via the information display device based on the identified results. The input is the output of step 2 (the identification result of the priority person). Based on this, the system informs mobile terminals and smart devices that priority users have the right to use the elevator, and encourages other users to use other means of transportation such as stairs. The output is the action displayed as an instruction message.

[0492] Step 4:

[0493] The user navigates based on directions provided by the server. User feedback is then input back into the system and analyzed by an information processing device. The input is user feedback data. This feedback is used to evaluate the validity and convenience of the directions and to improve the accuracy of future directions. The output is the analysis result, which contributes to the continuous improvement of the system.

[0494] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0495] This invention is a system that provides priority to elevator users, and by combining it with an emotion engine, it enables guidance that also takes into account the emotional state of the user. This system consists of an image acquisition device, an information processing device, an information display device, and an emotion engine.

[0496] The terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage and activates an AI model to recognize specific individuals. The analysis uses specific algorithms to determine whether the person is pregnant or using a stroller, wheelchair, or crutches.

[0497] The server determines whether a particular person needs priority use of the elevator or escalator. Based on this determination, the server sends and displays a message to other users via an information display device, encouraging them to use the stairs or escalator.

[0498] Furthermore, this system incorporates an emotion engine to recognize the user's emotional state. The server analyzes the user's facial expressions and body movements from the acquired video, and the emotion engine determines the user's stress and anxiety levels. Based on this information, the server can dynamically adjust the content and priority of the guidance it provides.

[0499] For example, if a user is showing signs of high stress, the server can use the emotion engine's assessment to avoid prioritizing the use of stairs or mobility devices, and instead guide the user towards using an elevator more smoothly. This ensures that elevators are available to those who need them, while taking into consideration the user's emotional state.

[0500] As a concrete example, if a terminal identifies a pregnant user exhibiting stress in front of an elevator during peak hours, and the emotion engine determines that the user has a high stress level, the server will guide this user to use a priority elevator and recommend alternative transportation methods to other users. This allows for flexible responses tailored to user preferences even during peak hours, improving overall efficiency.

[0501] The following describes the processing flow.

[0502] Step 1:

[0503] The terminal uses an image acquisition device installed around the elevator to capture video footage of the surroundings.

[0504] Step 2:

[0505] The terminal sends the captured video data to the server, which then activates an AI model for video analysis.

[0506] Step 3:

[0507] The server analyzes the person in the video to determine if they are pregnant or using a stroller, wheelchair, or crutches.

[0508] Step 4:

[0509] The server recognizes a specific person and evaluates whether that person needs priority use of the elevator.

[0510] Step 5:

[0511] Based on the recognition results, the server sends a message to the terminal encouraging other users to use the stairs or escalator if the user should have priority. The terminal then displays this message on its information display device.

[0512] Step 6:

[0513] The device uses an emotion engine to analyze the user's facial expressions from video and measure their stress and anxiety levels.

[0514] Step 7:

[0515] The server receives the results from the emotion engine and dynamically adjusts the guidance content based on the user's emotional state. For example, if the user is emotionally unstable and stressed, a special guidance message will encourage them to use the elevator.

[0516] Step 8:

[0517] The terminal displays a pre-arranged message from the server on the information display device, informing the user whether to use the elevator or choose an alternative mode of transportation.

[0518] Step 9:

[0519] If the user is inside the elevator, the device will capture video again inside the elevator and send it to the server.

[0520] Step 10:

[0521] The server analyzes the video footage inside the elevator and, if there is a user who should give way, sends a message to the terminal prompting them to get off on the next floor.

[0522] Step 11:

[0523] The terminal displays a message on the information display device inside the elevator, providing passengers with appropriate guidance for disembarking.

[0524] Step 12:

[0525] The server provides a feedback function to collect user opinions and use them to improve the accuracy of the guidance provided.

[0526] (Example 2)

[0527] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0528] Conventional mobility systems have insufficient consideration for users' physical conditions, making it difficult to prioritize specific individuals. Furthermore, they lacked the ability to provide guidance that took users' emotional states into account, resulting in insufficient effective responses to alleviate user stress and anxiety. It is necessary to improve this situation and provide users with more comfortable and efficient means of transportation.

[0529] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0530] In this invention, the server includes means for acquiring video footage of the area around the mobile device using an image acquisition device, means for analyzing the video footage to determine the need for priority use by a specific person, and means for evaluating the user's emotional state using emotion analysis technology and adjusting the guidance content. This optimizes the use of the mobile device based on the user's physical and emotional state, enabling comfortable and efficient guidance.

[0531] An "image acquisition device" is a device that captures video footage of the area around a mobile device and collects that data.

[0532] An "information processing device" is a computer that analyzes acquired video data to determine the priority use necessity and emotional state of a specific person.

[0533] An "information display device" is a display device that provides guidance to other users based on analysis results.

[0534] "Emotional analysis technology" is a technology that analyzes the user's facial expressions and posture from video data to evaluate their emotional state, such as stress and anxiety.

[0535] "Priority use necessity" refers to the criteria used to determine whether a particular person should have priority use of a mobility device.

[0536] "Adjusting the information provided" refers to the process of dynamically changing the information presented according to the user's emotional state and priorities.

[0537] This invention provides a system that enables priority guidance for specific users and considers the emotional state of users when using a mobile device. First, a terminal uses an image acquisition device installed around the mobile device to acquire video in real time. The acquired video data is transmitted to a server via the internet.

[0538] The server activates an AI model for image processing to analyze the received video data. This model uses a generative AI model to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users, etc.). This model includes algorithms such as face recognition and object detection.

[0539] Furthermore, the server uses emotion analysis technology to evaluate the user's emotional state. Specifically, it analyzes the user's facial expressions and movements from the video, and the emotion engine determines the level of stress and anxiety.

[0540] Based on these analysis results, the server dynamically generates and displays messages via an information display device encouraging other users to use stairs or other means of transportation. For example, if the server is concerned about a pregnant woman with a high stress level, it will prioritize guiding her to use transportation devices, while providing other users with a message such as, "Please use the stairs for the sake of the pregnant woman."

[0541] A concrete example of a prompt sentence for the generative AI model and emotion engine in this invention would be, "Please describe how the guidance system works to reduce stress for pregnant users during peak hours." In this way, the system supports the use of the optimal mode of transportation and provides comfortable guidance based on the user's physical and emotional state.

[0542] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0543] Step 1:

[0544] The terminal uses an image acquisition device to capture video footage of the surroundings of the mobile device. This video data is acquired in real time and transmitted to the server. The input is video footage from surrounding cameras, and the output is video data that can be processed.

[0545] Step 2:

[0546] The server activates an AI model using the received video data. The AI ​​model analyzes the input video data to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users). During this process, data processing such as face recognition and object detection is used to output the results of the person identification.

[0547] Step 3:

[0548] The server uses emotion analysis technology to analyze the user's facial and body movements. Using video data obtained in the previous stage as input, it calculates stress and anxiety levels. During this process, the emotion engine evaluates the user's emotional state, and an emotion score is obtained as output.

[0549] Step 4:

[0550] The server generates guidance messages to be displayed on the information display device based on the determined priority and emotional state. The inputs are the person's priority and emotional score, and the output is a dynamically adjusted guidance message. For example, a pregnant woman experiencing high stress will receive the message, "Please use this service with priority."

[0551] Step 5:

[0552] The terminal displays the generated guidance message on an information display device, encouraging other users to use stairs or other means of transportation. The input is the generated message, and the output is the display of visual guidance. This ensures appropriate navigation guidance.

[0553] (Application Example 2)

[0554] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0555] When using elevators, it is especially important to assign appropriate priorities to users in crowded environments or those experiencing physical or mental strain, and to support their movement safely and efficiently. However, conventional technology is insufficient in providing guidance that takes into account the physical condition and emotional state of users, which can lead to user dissatisfaction and confusion. Therefore, improving user satisfaction and operational efficiency in elevator usage scenarios is a challenge.

[0556] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0557] In this invention, the server includes means for acquiring video footage of the area around the elevator using an image acquisition device, means for analyzing the user's facial expressions and movements from the video footage and evaluating their stress level using an emotion engine, means for dynamically adjusting the guidance content based on the stress level evaluation and prioritizing the use of the elevator, and means for generating information to be displayed on a smart device. This makes it possible to guide users to use the elevator efficiently based on priority while reducing their burden.

[0558] An "image acquisition device" is a device used to capture video footage of the area around an elevator and transmits the acquired video data to an information processing device.

[0559] An "information processing device" is a device that analyzes acquired video footage to determine whether a particular user requires priority use of the elevator.

[0560] An "information display device" is a device that provides visual guidance from an information processing device to other users, encouraging them to use stairs, mobility facilities, and other such equipment.

[0561] An "emotion engine" is software or a system used to analyze a user's facial expressions and movements and evaluate their emotional state.

[0562] A "smart device" is an electronic device that a user can wear, which displays information in real time and guides the user's actions.

[0563] This invention is a system that provides priority to elevator users and guides them while considering their emotional state. First, image acquisition equipment installed around the elevator acquires video footage. This video footage is transmitted to an information processing device, where specific analysis is performed. This analysis uses specific algorithms and an emotion engine to determine whether the user is pregnant or using a stroller or wheelchair.

[0564] Based on this analysis, the information processing device determines whether a particular user needs priority use of the elevator. The server then uses this determination to provide guidance to other users through an information display device. In this process, the information display device shows other users a message encouraging them to use stairs or other mobility equipment.

[0565] In addition, the system analyzes the user's facial expressions and movements from the acquired video footage to assess their emotional state. To perform this assessment, an emotion engine is activated to determine the user's stress level. Based on this stress level, the server adjusts the guidance content in real time.

[0566] As a concrete example, consider a user using a smart device in a shopping mall. If this user is pushing a stroller and the emotion engine detects a high stress level, the server will prompt the user to prioritize using the elevator. Meanwhile, other users will be guided to use the escalator, thus ensuring a smoother overall flow of traffic.

[0567] By using a generative AI model, we aim to improve the accuracy of guidance and enhance user satisfaction. An example of a prompt to the generative AI model would be: "Determine whether the user wearing a smart device is experiencing stress, and if stress levels are high, suggest a method to guide them to the elevator first."

[0568] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0569] Step 1:

[0570] The terminal acquires video footage of the area around the elevator. This video is captured through an image acquisition device and transmitted to an information processing device. The information processing device receives the video data as input.

[0571] Step 2:

[0572] The server analyzes the received video. First, a generative AI model is used to identify a specific person, and then a specific algorithm is used to determine whether that person is pregnant or using a stroller or wheelchair. The input is video data, and the output generates identification information and priority ratings for the specific person.

[0573] Step 3:

[0574] The emotion engine is activated based on the identification information acquired by the server. It analyzes facial expressions and body movements obtained from the video to determine the user's stress level. The input is again video data, and the output is data related to the emotional state.

[0575] Step 4:

[0576] The server uses the determined stress level and priority information to generate guidance messages for information display devices. The input is stress level and priority information, and the output is a guidance message that will be presented to other users.

[0577] Step 5:

[0578] The server generates guidance messages and sends them to the smart device, encouraging the user to use stairs or mobility facilities. At this stage, the generated messages are displayed to guide the user's actions. The input is the guidance message, and the output is the display on the smart device.

[0579] This series of processes streamlines the overall flow and allows for efficient and considerate guidance to be provided to users.

[0580] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0581] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0582] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0583] [Fourth Embodiment]

[0584] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0585] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0586] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0587] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0588] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0589] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0590] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0591] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0592] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0593] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0594] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0595] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0596] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0597] This invention is a system that optimizes the use of elevators and assists in ensuring that those who need them can use them preferentially. The system consists of an image acquisition device, an information processing device, and an information display device. The following describes a specific embodiment for carrying out the invention.

[0598] In this system, the terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage to identify specific individuals. A specific algorithm is used for the analysis to identify people who require priority use of the elevator, including pregnant women and wheelchair users.

[0599] Based on this information, the server determines who can use the elevator and displays the result on the terminal via an information display device. Specifically, other users who are determined to be able to use the stairs or escalator will see a message on their screen encouraging them to use that mode of transportation.

[0600] On the other hand, if there is a user on the elevator, the server re-analyzes the video feed from inside the elevator. Based on this analysis, the server will, if necessary, send a message to the terminal informing it of the next floor's disembarkation and indicating the possibility that other passengers may yield the elevator.

[0601] This embodiment provides an environment where people who need it can use the elevator smoothly. The system also collects user feedback and uses that data to improve the accuracy of its analysis algorithms. Users can provide feedback on the appropriateness of the guidance through a dedicated screen, which contributes to system improvement.

[0602] As a concrete example, the device can identify pregnant individuals among those waiting in front of elevators during the morning rush hour and guide them to use the elevator first, while encouraging other users to use the stairs. This ensures that elevators are used efficiently even in crowded situations, allowing those who need them to move quickly.

[0603] The following describes the processing flow.

[0604] Step 1:

[0605] The terminal captures images of the surroundings through an image acquisition device installed at the entrance of the elevator.

[0606] Step 2:

[0607] The device sends the captured video data to the server. The server then activates an AI model to analyze the received video.

[0608] Step 3:

[0609] The server uses an AI model to analyze people in the video to determine if they are pregnant and whether they are using strollers, wheelchairs, crutches, etc.

[0610] Step 4:

[0611] Based on the analysis results, the server determines whether a particular person needs priority use of the elevator.

[0612] Step 5:

[0613] Based on the determination result, the server sends a message to the terminal encouraging other users to use the stairs or escalator.

[0614] Step 6:

[0615] The terminal displays messages received from the server on an information display device and guides the user to change their travel route.

[0616] Step 7:

[0617] If a user is inside the elevator, a terminal inside the elevator sends video of the current passenger status to the server.

[0618] Step 8:

[0619] The server analyzes the video footage from inside the elevator to determine if there is a user who can give up their seat.

[0620] Step 9:

[0621] If the server determines that it can yield, it sends a message to the terminal inside the elevator prompting passengers to disembark on the next floor.

[0622] Step 10:

[0623] The terminal displays a message on the information display device inside the elevator, informing passengers to disembark.

[0624] (Example 1)

[0625] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0626] The aim is to provide a method that allows necessary users to reach their destinations smoothly in situations where efficient use of elevators is difficult. Furthermore, it aims to improve overall travel efficiency by providing flexible guidance tailored to usage conditions.

[0627] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0628] In this invention, the server includes means for acquiring images using an imaging device that captures the surrounding area; means for utilizing an information processing system that analyzes the images to determine whether a particular individual requires preferential use; and means for collecting user feedback through a dedicated interface and supporting the improvement of the accuracy of the analysis algorithm using that data. This makes it possible to provide an environment in which individuals who need it can use the elevator preferentially, thereby improving overall mobility efficiency.

[0629] "Condition of the surrounding area" refers to the physical environment near the elevator and the movements of people within it, and this information is acquired by the imaging device.

[0630] An "imaging device" refers to a camera or other video capture device used to acquire video data in real time.

[0631] "Analyzing video" refers to processing acquired video data and performing computational operations to identify individuals who meet specific characteristics or conditions. This process utilizes algorithms and machine learning techniques.

[0632] An "information processing system" refers to a system consisting of computing resources and software for analyzing video, evaluating data, and generating judgment results.

[0633] "Specific individuals" refers to people who require priority use of elevators, including, for example, pregnant women or those who require physical assistance.

[0634] A "display device" refers to a monitor or screen used to visually present information, and is used to display guidance messages to users.

[0635] "Collecting opinions through a dedicated interface" refers to obtaining information through a software or hardware interface specifically designed to receive user feedback.

[0636] "Improving the accuracy of analysis algorithms" refers to the process of improving the system's judgment and recognition capabilities by using newly collected data and feedback.

[0637] This invention implements a system that enables the efficient use of elevators. The system mainly consists of terminals, servers, and users.

[0638] Terminal role:

[0639] The terminal is equipped with imaging devices to acquire video footage of the area around and inside the elevator. This includes high-resolution cameras and video capture devices. The terminal uses these devices to acquire video data in real time and transmits it to the server in digital format.

[0640] Server role:

[0641] The server is responsible for analyzing the received video data. The generative AI model used here utilizes image recognition technology and implements algorithms to identify specific individuals who require priority access, such as pregnant women or wheelchair users. Frameworks such as TensorFlow and PyTorch are used in this analysis process. Based on the analysis results, the server determines the user's priority and sends data to the terminal to display the necessary information via an information display device.

[0642] Furthermore, the server collects user feedback and uses it to improve the accuracy of the analysis algorithm. This process involves analyzing the feedback data and generating prompts to update the model. An example of a specific prompt is, "Based on user feedback data, please propose solutions to improve the accuracy of identifying priority elevator users."

[0643] User roles:

[0644] Users act based on information presented by the system. Users with priority access to the elevator are guided to move, while other users are encouraged to use alternative means of transport. Users also provide feedback through a dedicated interface, contributing to system improvements in the process.

[0645] A notable feature of this system is its ability to monitor the situation in real time and provide appropriate instructions to individuals, enabling efficient use of elevators even during peak hours.

[0646] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0647] Step 1:

[0648] The terminal acquires video footage of the area around and inside the elevator using an imaging device. Specifically, the terminal activates a high-resolution camera and captures video in real time. The acquired video data is converted into a digital format and sent to the server as is.

[0649] Step 2:

[0650] The server analyzes the video data received from the terminal. The server uses a generative AI model to perform image recognition processing on the digital video data received as input. Frameworks such as TensorFlow are used for data calculations to identify specific individuals in the video (e.g., pregnant women or wheelchair users). The output of this process is a list of identified priority users.

[0651] Step 3:

[0652] The server generates guidance information for users based on the analysis results. The server evaluates the list of priority users and creates guidance messages to send to terminals via display devices. Specifically, the guidance created includes recommendations for priority users to use elevators and messages encouraging other users to use stairs or escalators.

[0653] Step 4:

[0654] The terminal displays the guidance information received from the server on the display device. Specifically, the terminal displays the message in the appropriate location on the screen, providing the user with visual information. At this time, user profile information is also used to determine which user should view which guidance.

[0655] Step 5:

[0656] Users act based on the displayed instructions. Priority users use the elevator, while other users use the recommended alternative. Furthermore, feedback on the appropriateness of the instructions can be provided through a dedicated interface.

[0657] Step 6:

[0658] The server analyzes the feedback data received from the user. The server receives the feedback data as input and generates prompt statements based on this data to improve the model's accuracy. These prompt statements are used to improve the algorithm in the next iteration, contributing to overall system performance improvement.

[0659] (Application Example 1)

[0660] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] In public and commercial facilities, a problem exists where people who require priority access to elevators and escalators cannot receive service quickly and smoothly, resulting in reduced convenience. To address this issue, it is necessary to provide optimal guidance to users and improve the overall operational efficiency of elevators and escalators.

[0662] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0663] In this invention, the server includes means for acquiring surrounding images using an image acquisition device, means for determining whether priority use is necessary using an information processing device, and means for providing guidance on a mobile terminal via an information display device. This makes it possible for people who require priority to quickly use the elevator, provide appropriate navigation guidance to other users, and improve overall travel efficiency.

[0664] A "video acquisition device" is a device that collects visual information from the surroundings, and uses cameras or other devices to visualize the situation in front of and behind the elevator.

[0665] An "information processing device" is a computing device that analyzes acquired video data and determines whether a particular person needs priority use of the elevator.

[0666] An "information display device" is a device that provides guidance information to mobile terminals and smart devices and displays appropriate instructions to users.

[0667] A "smart device" is an electronic device that uses mobile terminals and related technologies to provide information necessary for using elevators.

[0668] A "generative AI model" is an artificial intelligence computational model used to support decision-making in video analysis and to identify specific users.

[0669] This invention is a system for optimizing the use of elevators and effectively assisting users who require priority. The system primarily comprises hardware such as a video acquisition device, an information processing device, and an information display device, and utilizes a generative AI model as software.

[0670] The server uses a video acquisition device to collect images of the area around the elevator. This video data is analyzed by an information processing device, and a generative AI model is used to identify specific users, such as pregnant women or wheelchair users. This allows the system to identify individuals who require priority access and provide guidance information to their mobile devices via smart devices.

[0671] The terminal provides optimal navigation guidance to other users via an information display device. This improves the efficiency of elevator use and enhances convenience. The system has a mechanism to continuously improve the accuracy of guidance by collecting and analyzing user feedback using the information display device.

[0672] As a concrete example, if a pregnant woman tries to use an elevator in a shopping mall, the system can automatically prioritize her and direct others to use the stairs. This guidance is trained to effectively operate a generative AI model through explicitly stated prompts. An example prompt would be, "Please describe in detail the system that uses cameras to capture people waiting for elevators in a shopping mall and identifies and displays those who need priority access."

[0673] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0674] Step 1:

[0675] The server captures surrounding video data using a video acquisition device. In this process, cameras monitor the front and back of the elevator, collecting images in real time. The input is video data from the cameras, and the output is video frames that can be analyzed. The obtained data is sent to an information processing device for further processing.

[0676] Step 2:

[0677] The server receives video frames transmitted by the information processing device and performs data analysis using a generative AI model. The input is the video frames acquired in step 1. The generative AI model performs a process to identify specific individuals (e.g., pregnant women or wheelchair users). The output is the result that individuals requiring priority use have been identified. This identification result is then passed on to the next step within the system.

[0678] Step 3:

[0679] The information processing device displays instructions on the terminal via the information display device based on the identified results. The input is the output of step 2 (the identification result of the priority person). Based on this, the system informs mobile terminals and smart devices that priority users have the right to use the elevator, and encourages other users to use other means of transportation such as stairs. The output is the action displayed as an instruction message.

[0680] Step 4:

[0681] The user navigates based on directions provided by the server. User feedback is then input back into the system and analyzed by an information processing device. The input is user feedback data. This feedback is used to evaluate the validity and convenience of the directions and to improve the accuracy of future directions. The output is the analysis result, which contributes to the continuous improvement of the system.

[0682] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0683] This invention is a system that provides priority to elevator users, and by combining it with an emotion engine, it enables guidance that also takes into account the emotional state of the user. This system consists of an image acquisition device, an information processing device, an information display device, and an emotion engine.

[0684] The terminal first acquires video footage of the area around the elevator. The acquired video is sent to a server, which analyzes the footage and activates an AI model to recognize specific individuals. The analysis uses specific algorithms to determine whether the person is pregnant or using a stroller, wheelchair, or crutches.

[0685] The server determines whether a particular person needs priority use of the elevator or escalator. Based on this determination, the server sends and displays a message to other users via an information display device, encouraging them to use the stairs or escalator.

[0686] Furthermore, this system incorporates an emotion engine to recognize the user's emotional state. The server analyzes the user's facial expressions and body movements from the acquired video, and the emotion engine determines the user's stress and anxiety levels. Based on this information, the server can dynamically adjust the content and priority of the guidance it provides.

[0687] For example, if a user is showing signs of high stress, the server can use the emotion engine's assessment to avoid prioritizing the use of stairs or mobility devices, and instead guide the user towards using an elevator more smoothly. This ensures that elevators are available to those who need them, while taking into consideration the user's emotional state.

[0688] As a concrete example, if a terminal identifies a pregnant user exhibiting stress in front of an elevator during peak hours, and the emotion engine determines that the user has a high stress level, the server will guide this user to use a priority elevator and recommend alternative transportation methods to other users. This allows for flexible responses tailored to user preferences even during peak hours, improving overall efficiency.

[0689] The following describes the processing flow.

[0690] Step 1:

[0691] The terminal uses an image acquisition device installed around the elevator to capture video footage of the surroundings.

[0692] Step 2:

[0693] The terminal sends the captured video data to the server, which then activates an AI model for video analysis.

[0694] Step 3:

[0695] The server analyzes the person in the video to determine if they are pregnant or using a stroller, wheelchair, or crutches.

[0696] Step 4:

[0697] The server recognizes a specific person and evaluates whether that person needs priority use of the elevator.

[0698] Step 5:

[0699] Based on the recognition results, the server sends a message to the terminal encouraging other users to use the stairs or escalator if the user should have priority. The terminal then displays this message on its information display device.

[0700] Step 6:

[0701] The device uses an emotion engine to analyze the user's facial expressions from video and measure their stress and anxiety levels.

[0702] Step 7:

[0703] The server receives the results from the emotion engine and dynamically adjusts the guidance content based on the user's emotional state. For example, if the user is emotionally unstable and stressed, a special guidance message will encourage them to use the elevator.

[0704] Step 8:

[0705] The terminal displays a pre-arranged message from the server on the information display device, informing the user whether to use the elevator or choose an alternative mode of transportation.

[0706] Step 9:

[0707] If the user is inside the elevator, the device will capture video again inside the elevator and send it to the server.

[0708] Step 10:

[0709] The server analyzes the video footage inside the elevator and, if there is a user who should give way, sends a message to the terminal prompting them to get off on the next floor.

[0710] Step 11:

[0711] The terminal displays a message on the information display device inside the elevator, providing passengers with appropriate guidance for disembarking.

[0712] Step 12:

[0713] The server provides a feedback function to collect user opinions and use them to improve the accuracy of the guidance provided.

[0714] (Example 2)

[0715] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0716] Conventional mobility systems have insufficient consideration for users' physical conditions, making it difficult to prioritize specific individuals. Furthermore, they lacked the ability to provide guidance that took users' emotional states into account, resulting in insufficient effective responses to alleviate user stress and anxiety. It is necessary to improve this situation and provide users with more comfortable and efficient means of transportation.

[0717] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0718] In this invention, the server includes means for acquiring video footage of the area around the mobile device using an image acquisition device, means for analyzing the video footage to determine the need for priority use by a specific person, and means for evaluating the user's emotional state using emotion analysis technology and adjusting the guidance content. This optimizes the use of the mobile device based on the user's physical and emotional state, enabling comfortable and efficient guidance.

[0719] An "image acquisition device" is a device that captures video footage of the area around a mobile device and collects that data.

[0720] An "information processing device" is a computer that analyzes acquired video data to determine the priority use necessity and emotional state of a specific person.

[0721] An "information display device" is a display device that provides guidance to other users based on analysis results.

[0722] "Emotional analysis technology" is a technology that analyzes the user's facial expressions and posture from video data to evaluate their emotional state, such as stress and anxiety.

[0723] "Priority use necessity" refers to the criteria used to determine whether a particular person should have priority use of a mobility device.

[0724] "Adjusting the information provided" refers to the process of dynamically changing the information presented according to the user's emotional state and priorities.

[0725] This invention provides a system that enables priority guidance for specific users and considers the emotional state of users when using a mobile device. First, a terminal uses an image acquisition device installed around the mobile device to acquire video in real time. The acquired video data is transmitted to a server via the internet.

[0726] The server activates an AI model for image processing to analyze the received video data. This model uses a generative AI model to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users, etc.). This model includes algorithms such as face recognition and object detection.

[0727] Furthermore, the server uses emotion analysis technology to evaluate the user's emotional state. Specifically, it analyzes the user's facial expressions and movements from the video, and the emotion engine determines the level of stress and anxiety.

[0728] Based on these analysis results, the server dynamically generates and displays messages via an information display device encouraging other users to use stairs or other means of transportation. For example, if the server is concerned about a pregnant woman with a high stress level, it will prioritize guiding her to use transportation devices, while providing other users with a message such as, "Please use the stairs for the sake of the pregnant woman."

[0729] A concrete example of a prompt sentence for the generative AI model and emotion engine in this invention would be, "Please describe how the guidance system works to reduce stress for pregnant users during peak hours." In this way, the system supports the use of the optimal mode of transportation and provides comfortable guidance based on the user's physical and emotional state.

[0730] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0731] Step 1:

[0732] The terminal uses an image acquisition device to capture video footage of the surroundings of the mobile device. This video data is acquired in real time and transmitted to the server. The input is video footage from surrounding cameras, and the output is video data that can be processed.

[0733] Step 2:

[0734] The server activates an AI model using the received video data. The AI ​​model analyzes the input video data to identify people in the video and determine if they meet specific criteria (e.g., pregnant women, wheelchair users). During this process, data processing such as face recognition and object detection is used to output the results of the person identification.

[0735] Step 3:

[0736] The server uses emotion analysis technology to analyze the user's facial and body movements. Using video data obtained in the previous stage as input, it calculates stress and anxiety levels. During this process, the emotion engine evaluates the user's emotional state, and an emotion score is obtained as output.

[0737] Step 4:

[0738] The server generates guidance messages to be displayed on the information display device based on the determined priority and emotional state. The inputs are the person's priority and emotional score, and the output is a dynamically adjusted guidance message. For example, a pregnant woman experiencing high stress will receive the message, "Please use this service with priority."

[0739] Step 5:

[0740] The terminal displays the generated guidance message on an information display device, encouraging other users to use stairs or other means of transportation. The input is the generated message, and the output is the display of visual guidance. This ensures appropriate navigation guidance.

[0741] (Application Example 2)

[0742] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0743] When using elevators, it is especially important to assign appropriate priorities to users in crowded environments or those experiencing physical or mental strain, and to support their movement safely and efficiently. However, conventional technology is insufficient in providing guidance that takes into account the physical condition and emotional state of users, which can lead to user dissatisfaction and confusion. Therefore, improving user satisfaction and operational efficiency in elevator usage scenarios is a challenge.

[0744] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0745] In this invention, the server includes means for acquiring video footage of the area around the elevator using an image acquisition device, means for analyzing the user's facial expressions and movements from the video footage and evaluating their stress level using an emotion engine, means for dynamically adjusting the guidance content based on the stress level evaluation and prioritizing the use of the elevator, and means for generating information to be displayed on a smart device. This makes it possible to guide users to use the elevator efficiently based on priority while reducing their burden.

[0746] An "image acquisition device" is a device used to capture video footage of the area around an elevator and transmits the acquired video data to an information processing device.

[0747] An "information processing device" is a device that analyzes acquired video footage to determine whether a particular user requires priority use of the elevator.

[0748] An "information display device" is a device that provides visual guidance from an information processing device to other users, encouraging them to use stairs, mobility facilities, and other such equipment.

[0749] An "emotion engine" is software or a system used to analyze a user's facial expressions and movements and evaluate their emotional state.

[0750] A "smart device" is an electronic device that a user can wear, which displays information in real time and guides the user's actions.

[0751] This invention is a system that provides priority to elevator users and guides them while considering their emotional state. First, image acquisition equipment installed around the elevator acquires video footage. This video footage is transmitted to an information processing device, where specific analysis is performed. This analysis uses specific algorithms and an emotion engine to determine whether the user is pregnant or using a stroller or wheelchair.

[0752] Based on this analysis, the information processing device determines whether a particular user needs priority use of the elevator. The server then uses this determination to provide guidance to other users through an information display device. In this process, the information display device shows other users a message encouraging them to use stairs or other mobility equipment.

[0753] In addition, the system analyzes the user's facial expressions and movements from the acquired video footage to assess their emotional state. To perform this assessment, an emotion engine is activated to determine the user's stress level. Based on this stress level, the server adjusts the guidance content in real time.

[0754] As a concrete example, consider a user using a smart device in a shopping mall. If this user is pushing a stroller and the emotion engine detects a high stress level, the server will prompt the user to prioritize using the elevator. Meanwhile, other users will be guided to use the escalator, thus ensuring a smoother overall flow of traffic.

[0755] By using a generative AI model, we aim to improve the accuracy of guidance and enhance user satisfaction. An example of a prompt to the generative AI model would be: "Determine whether the user wearing a smart device is experiencing stress, and if stress levels are high, suggest a method to guide them to the elevator first."

[0756] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0757] Step 1:

[0758] The terminal acquires video footage of the area around the elevator. This video is captured through an image acquisition device and transmitted to an information processing device. The information processing device receives the video data as input.

[0759] Step 2:

[0760] The server analyzes the received video. First, a generative AI model is used to identify a specific person, and then a specific algorithm is used to determine whether that person is pregnant or using a stroller or wheelchair. The input is video data, and the output generates identification information and priority ratings for the specific person.

[0761] Step 3:

[0762] The emotion engine is activated based on the identification information acquired by the server. It analyzes facial expressions and body movements obtained from the video to determine the user's stress level. The input is again video data, and the output is data related to the emotional state.

[0763] Step 4:

[0764] The server uses the determined stress level and priority information to generate guidance messages for information display devices. The input is stress level and priority information, and the output is a guidance message that will be presented to other users.

[0765] Step 5:

[0766] The server generates guidance messages and sends them to the smart device, encouraging the user to use stairs or mobility facilities. At this stage, the generated messages are displayed to guide the user's actions. The input is the guidance message, and the output is the display on the smart device.

[0767] This series of processes streamlines the overall flow and allows for efficient and considerate guidance to be provided to users.

[0768] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0769] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0770] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0771] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0772] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0773] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0774] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0775] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0776] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0777] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0778] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0779] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0780] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0781] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0782] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0783] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0784] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0785] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0786] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0787] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0788] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0789] The following is further disclosed regarding the embodiments described above.

[0790] (Claim 1)

[0791] An image acquisition device provides a means for acquiring video footage of the area around the elevator,

[0792] A method using an information processing device that analyzes the video and determines whether a specific person needs priority use of the elevator,

[0793] A means of providing guidance to other users via an information display device, based on the judgment result, encouraging them to use stairs or mobility devices,

[0794] A system that includes this.

[0795] (Claim 2)

[0796] The system according to claim 1, wherein the information processing device provides guidance to other passengers inside the elevator, prompting them to disembark at the next floor.

[0797] (Claim 3)

[0798] The system according to claim 1, wherein the information processing device provides priority guidance to specific individuals who require an elevator, and includes means for collecting and analyzing user feedback for the purpose of improving the accuracy of the guidance.

[0799] "Example 1"

[0800] (Claim 1)

[0801] A means for acquiring images using an imaging device that acquires the conditions of the surrounding area,

[0802] A method using an information processing system that analyzes the video to determine whether a specific individual needs priority access,

[0803] A means of providing guidance to other users via a display device, suggesting the use of alternative means based on the judgment result,

[0804] A means to collect user feedback through a dedicated interface and support the improvement of the accuracy of analysis algorithms using that data,

[0805] A system that includes this.

[0806] (Claim 2)

[0807] The system according to claim 1, wherein the information processing system issues instructions to other travelers within the system to urge them to leave at the next level.

[0808] (Claim 3)

[0809] The system according to claim 1, wherein the information processing system provides preferential guidance to a specific individual and includes means for collecting and analyzing user feedback for the purpose of improving the accuracy of such guidance.

[0810] "Application Example 1"

[0811] (Claim 1)

[0812] A means for acquiring images of the area around the elevator using an image acquisition device,

[0813] A method using an information processing device that analyzes the image and determines whether a specific person needs priority use of the elevator,

[0814] A means of providing guidance to other users via an information display device, based on the judgment result, encouraging them to use stairs or other means of transportation,

[0815] A means of providing guidance to priority users on mobile devices using smart devices,

[0816] A system that includes this.

[0817] (Claim 2)

[0818] The system according to claim 1, further comprising an information processing device providing guidance to other passengers inside the elevator prompting them to disembark at the next floor, and guidance using a smart device.

[0819] (Claim 3)

[0820] The system according to claim 1, comprising an information processing device providing priority guidance to specific individuals requiring elevators, means for collecting and analyzing user feedback for the purpose of improving the accuracy of the guidance, and analysis based on a generated AI model.

[0821] "Example 2 of combining an emotion engine"

[0822] (Claim 1)

[0823] A means for acquiring video footage of the area around a mobile device using an image acquisition device,

[0824] A method using an information processing device that analyzes the video and determines the need for priority use of a specific person's mobility device,

[0825] A means of providing guidance to other users via an information display device based on the judgment result, encouraging them to use stairs or other means of transportation,

[0826] A means of evaluating the user's emotional state using emotion analysis technology and adjusting the guidance content based on the evaluation results,

[0827] A system that includes this.

[0828] (Claim 2)

[0829] The system according to claim 1, wherein the information processing device provides guidance to passengers inside the mobile device, prompting them to disembark at the next floor.

[0830] (Claim 3)

[0831] The system according to claim 1, comprising: an information processing device providing priority guidance to specific individuals requiring a mobile device; means for collecting and analyzing user feedback for the purpose of improving guidance accuracy; and means for optimizing guidance by reflecting the user's emotional state.

[0832] "Application example 2 when combining with an emotional engine"

[0833] (Claim 1)

[0834] A means of acquiring video footage of the area around the elevator using an image acquisition device,

[0835] A method using an information processing device that analyzes the video and determines whether a particular user requires priority use of the elevator,

[0836] A means of providing guidance to other users via information display devices to encourage them to use stairs or mobility facilities,

[0837] A method for analyzing the user's facial expressions and movements from video and evaluating their stress level using an emotion engine,

[0838] A means of dynamically adjusting guidance content based on stress level assessment and prioritizing the use of elevators,

[0839] A means for generating information to be displayed on a smart device,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, wherein the information processing device provides guidance to other passengers inside the elevator, prompting them to disembark at the next floor, and displays this information on a smart device.

[0843] (Claim 3)

[0844] The system according to claim 1, comprising an information processing device that provides priority guidance to specific users who require an elevator, and means for collecting and analyzing user feedback for the purpose of improving the accuracy of the guidance, and using this feedback to train a generating AI to optimize the guidance. [Explanation of Symbols]

[0845] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An image acquisition device provides a means for acquiring video footage of the area around the elevator, A method using an information processing device that analyzes the video and determines whether a specific person needs priority use of the elevator, A means of providing guidance to other users via an information display device, based on the judgment result, encouraging them to use stairs or mobility devices, A system that includes this.

2. The system according to claim 1, wherein the information processing device provides guidance to other passengers inside the elevator, prompting them to disembark at the next floor.

3. The system according to claim 1, wherein the information processing device provides priority guidance to specific individuals who require an elevator, and includes means for collecting and analyzing user feedback for the purpose of improving the accuracy of the guidance.

Images