Utilization promotion system and its control method

Abstract

This technology utilizes information obtained from cameras installed in elevators to promote the efficient use of buildings. [Solution] Based on the image of passenger 5 taken by camera 19, server 80 determines the area information of a recommended area to recommend passenger 5 to use from among the area information of multiple areas. The notification device notifies the area information of the recommended area. When server 80 determines that passenger 5 has indicated their intention to use the recommended area after the area information of the recommended area has been notified by the notification device, it sends a command to the control device 20 of elevator 1 to register a call for the floor where the recommended area is located in elevator car 10.

Description

Technical Field

[0001] The present disclosure relates to a usage promotion system that promotes the use of a plurality of areas provided in a building, and a control method thereof.

Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2008-133097 (Patent Document 1) discloses a technology that sequentially displays advertisement images on a display unit provided in an elevator car, and grants points when a user causes a reader / writer to read an ID card. In addition, Japanese Unexamined Patent Application Publication No. 2011-170117 (Patent Document 2) discloses a technology in which a content server distributes advertisement information suitable for each user to an advertisement terminal in accordance with personal information contained in a terminal database.

[0003] On the other hand, a camera may be installed in an elevator car or at a landing. Information obtained from this camera is used for limited applications such as crime prevention.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of Invention

Problems to be Solved by the Invention

[0005] As described above, since cameras installed for elevators are only used for limited applications, there exists a customer need to expand the range of utilization methods for information obtained from the cameras, for example, to utilize such information for applications such as advertisements for shops in the building.

[0006] This disclosure was made to solve the aforementioned problems, and its purpose is to provide a technology that can promote the use of a building by utilizing information obtained from a camera installed in an elevator. [Means for solving the problem]

[0007] The usage promotion system described in this disclosure is a system that promotes the use of multiple areas provided in a building. The usage promotion system comprises a camera, a server, and a notification device. The camera photographs passengers inside the elevator car installed in the building. Based on the images of passengers captured by the camera, the server determines the area information of a recommended area from among the area information of multiple areas to recommend to the passenger. The notification device notifies the passenger of the area information of the recommended area. When the server determines that the passenger has indicated their intention to use the recommended area after the area information of the recommended area has been notified by the notification device, it transmits a command to the elevator control device to register the car call for the floor where the recommended area is located.

[0008] The control method relating to this disclosure is a control method for a utilization promotion system that promotes the use of multiple areas provided in a building. The utilization promotion system comprises a camera that photographs passengers inside an elevator car installed in the building, a server, and a notification device. The control method comprises the steps of: the server determining area information of a recommended area to recommend to the passenger from among the area information of multiple areas based on the passenger image taken by the camera; the notification device notifying the area information of the recommended area; and when the server determines that the passenger has indicated their intention to use the recommended area after the notification device has notified the area information of the recommended area, the server transmitting a command to the elevator control device to register a car call for the floor where the recommended area is located. [Effects of the Invention]

[0009] According to this disclosure, information obtained from cameras installed in elevators can be used to promote the use of buildings. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the overall configuration of the utilization promotion system according to the first embodiment. [Figure 2] This figure shows an example of the hardware configuration of a system designed to promote usage. [Figure 3] This figure shows an example of an area database. [Figure 4] This is a diagram illustrating an example of an estimation result. [Figure 5] This is a diagram illustrating an example of how area information is displayed. [Figure 6] This is a flowchart of the processes performed by the utilization promotion system. [Figure 7] This figure shows an example of training data according to the second embodiment. [Figure 8] This is a diagram showing an example of the configuration of a learning device. [Figure 9] This is a flowchart showing the learning process of the learning device. [Figure 10] This figure shows an example of the configuration of the inference processing unit. [Figure 11] This is a flowchart illustrating the inference procedure performed by the inference processing unit. [Modes for carrying out the invention]

[0011] The embodiments will be described below with reference to the drawings. In the following description, identical parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0012] [First Embodiment] Figure 1 shows an example of the overall configuration of the utilization promotion system 100 according to the first embodiment. Figure 2 shows an example of the hardware configuration of the utilization promotion system 100.

[0013] As shown in Figure 1, the usage promotion system 100 is a system that promotes the usage of a plurality of areas (for example, stores, event holding areas, etc.) provided in a building (in the present embodiment, a commercial facility). The building is not limited to a commercial facility, and may be a station premises, an office building, or the like, and may be any building as long as it has some stores or holds events.

[0014] The usage promotion system 100 includes a server 80, a communication device 50, a control device 20 provided in an elevator 1 (Figure 2), a plurality of cameras 19, a user terminal 700 and an announcement device 72 as notification devices, and a store terminal 600. The communication device 50, the store terminal 600, and the user terminal 700 can communicate with the server 80.

[0015] The elevator 1 is installed in this commercial facility. The server 80 may be installed in the commercial facility where the elevator 1 is installed, or may be installed in the information center of a maintenance company that performs maintenance of the elevator 1.

[0016] The control device 20 is configured to be communicatively connectable to the server 80 via the communication device 50. The announcement device 72 and the information device 73 (display device) are provided in the landing and the car 10. The server 80 acquires image information from the camera 19 and audio information from a microphone (not shown) via the communication device 50 and the control device 20, and issues an announcement command (audio output command) to the announcement device 72 in the landing or the car 10, and can issue a display command (screen output command) to the information device 73 in the landing or the car 10. The information device 73 may be provided with an audio output function (the function of the announcement device 72).

[0017] Cameras 19 are installed inside the elevator car 10 and on the landing. The camera 19 installed in the car 10 photographs the passenger 5 inside the car 10. The camera 19 installed on the landing photographs the passenger 5 on the landing. In the example in Figure 1, passenger 5c is on the landing to board the car 10. Passenger 5c is carrying a user terminal 700. Passenger 5c is photographed by the camera 19 on the landing side, and the captured image is transmitted to the server 80.

[0018] Meanwhile, passengers 5a and 5b are inside the elevator car 10. Passengers 5a and 5b are a family with a parent and child. In this embodiment, passengers 5a to 5c are collectively referred to as "passenger 5". Passenger 5a is carrying a user terminal 700. Passengers 5a and 5b are photographed by the camera 19 on the elevator car 10 side, and the captured images are transmitted to the server 80.

[0019] In this embodiment, the server 80 provides passenger 5 with area information (store sale information, event information, etc.) within the commercial facility based on the characteristics of passenger 5 inferred from the passenger's image taken at the boarding area or inside the elevator car 10. If passenger 5 indicates their preference in response to this area information, the server 80 automatically registers a request for the elevator car to go to the floor where that area (store, event venue, etc.) is located. As a result, passenger 5 who has indicated their preference can disembark on the floor where that area is located.

[0020] In the example shown in Figure 1, passengers 5a and 5b (a parent and child) are guided to Food Court B on the 7th floor via voice output from the announcement device 72 and display on the information device 73. Such guidance may be given either by voice or display, and may not be given at the boarding area, but only inside the elevator car 10. In response, passenger 5a indicates their intention by raising their hand. This is captured by the camera 19 and transmitted to the server 80. If the server 80 determines that an indication of intent has been made, such as by raising a hand or speaking, it sends a command to the control device 20 to register a request for the elevator car to go to the 7th floor where Food Court B is located. Examples of verbal indications include "Yes," "I'm going," and "To the 7th floor." This voice is collected by the microphone and transmitted to the server 80. The server 80 analyzes this voice to determine whether an indication of intent has been made. The control device 20 registers the request for the elevator car to go to the 7th floor, and as a result, the elevator car 10 stops and opens its doors on the 7th floor. Additionally, passenger 5c at the boarding area is being directed to store A on the 5th floor. More details will follow later.

[0021] As shown in Figure 2, the elevator 1 consists of a control device 20 that controls the elevator 1, a hoisting machine (not shown), a car device 30 installed in the car 10, landing devices 40 installed at the landings of each floor from the first floor to the top floor, and various sensors and switches used in the elevator 1.

[0022] The control device 20 comprises a processing unit 21, a storage device 22, and a communication interface (IF) (not shown). These are interconnected via a bus so that they can communicate with each other.

[0023] The processing unit 21 is, for example, a processor such as a CPU (Central Processing Unit). The storage device 22 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and a non-volatile storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0024] The processing unit 21 loads programs stored in ROM or storage device into RAM and executes them to realize various functions of the control device 20. The processing unit 21 executes the process of controlling the elevator 1. ROM stores programs that describe the processing procedures for the processes executed by the control device 20. RAM is the workspace for the processor when executing programs and temporarily stores programs and data used when executing programs. The communication interface has the function of communicating with various devices connected to the control device 20.

[0025] The hoisting machine is a motor that drives the elevator car 10 of elevator 1 to raise and lower it. The car device 30 consists of various devices installed in the car 10, including a camera 19, an announcement device 72, and a destination floor button (car call button) (not shown) for registering the destination floor. The landing device 40 consists of various devices installed in the landings of each floor, including a camera 19 and a landing call button (not shown) for registering an upward or downward landing call.

[0026] The elevator car 10 is installed in an elevator shaft located within a commercial facility. The elevator car 10 moves up and down within the elevator shaft to travel between multiple floors. In this embodiment, the elevator car 10 can stop at each floor from the 1st floor to the top floor (for example, the 10th floor). A machine room is provided directly above the elevator shaft. The machine room is equipped with a hoisting machine, a control device 20, and a communication device 50. The server 80 may be located in the machine room or the manager's room. Multiple elevator cars 10 may be installed.

[0027] The control device 20 is connected to multiple cameras 19 installed in the landing and inside the elevator car 10, and is configured to acquire images captured by the multiple cameras 19. The control device 20 is also connected to an announcement device 72 installed in the landing and inside the elevator car 10, and can issue announcement commands to the announcement device 72. The announcement device 72 makes announcements (outputs voice) based on the announcement commands from the control device 20. The control device 20 is also connected to an information device 73 installed in the landing and inside the elevator car 10, and can issue display commands to the information device 73. The information device 73 displays information on the screen based on the display commands from the control device 20. The control device 20 can communicate with the server 80 via a communication device 50.

[0028] Server 80, like control device 20, includes a processing unit 81, a storage device 82, and a communication interface (not shown). The processing unit 81 is, for example, a processor (CPU). The storage device 82 is, for example, memory (ROM, RAM, etc.). These are connected to each other via a bus so that they can communicate with one another.

[0029] The server 80 processes the information obtained from the control device 20 via the communication device 50 using the processing unit 81, and transmits the processing result to the control device 20. The server 80 also stores the information obtained from the control device 20 in the storage device 82.

[0030] The communication device 50 transmits various information, such as elevator 1, acquired from the control device 20 to the server 80, and also transmits commands from the server 80 to the control device 20. The communication device 50 also includes a processor (CPU), memory (ROM, RAM, etc.), and a communication interface (not shown). These are connected to each other via a bus so that they can communicate with one another.

[0031] The user terminal 700 is a mobile terminal such as a smartphone used by passenger 5 of elevator 1, and comprises a processor (CPU) (not shown), memory (ROM, RAM, etc.), a communication interface, an input unit, and a display unit. The input unit receives input from passenger 5. The display unit displays various information. The input unit and display unit are configured, for example, as touch panels. The user terminal 700 can communicate with server 80.

[0032] The store terminal 600 is a terminal used by employees in an area (such as a store) within a commercial facility, and comprises a processor (CPU) (not shown), memory (ROM, RAM, etc.), a communication interface, an input unit, and a display unit. The input unit is, for example, a keyboard and a mouse. The display unit is, for example, a display.

[0033] The store terminal 600 can communicate with the server 80 and obtain information managed by the server 80. Store sales information, event information, etc., can be registered in the area database 91 (described later) via the input section of the store terminal 600. Furthermore, various information such as sales information and event information can be viewed on the display section of the store terminal 600.

[0034] Figure 3 shows an example of an area database 91. The storage device 82 stores an area database 91 in which area information for multiple areas within a commercial facility is recorded. Each of the area information entries for multiple areas includes discount information (store sale information) and event information for event spaces within that area.

[0035] Area information includes floor level, area, type, content, and characteristic classification. Here, "Area" indicates the section where the store is located or the section where the event is held, and includes the store name, event name, etc. "Floor level" indicates the floor on which the area is located. "Type" indicates the classification of the store / event, such as food and beverage or fashion. "Content" indicates the details of sales, etc., conducted at the store / event. "Characteristic classification" is the item corresponding to estimation result 92 in Figure 4 (described later).

[0036] For example, the area database 91 records the following as the first area information: floor "5th floor", area "Store A", type "Fashion 1", content "Sale", and feature classification "N001". This indicates that Store A, located on the 5th floor and classified as Fashion 1, is currently having a sale. This information can be entered from the store terminal 600 at Store A. The information displayed in Figure 5, described later, corresponds to the feature classification "N001".

[0037] In fashion, "category" refers to classifications such as women's luxury brands (Fashion 1), women's casual wear, men's luxury brands, and men's casual wear. For example, if passenger 5 is a woman and her clothing and belongings (accessories) fall under the category of women's luxury brands (Fashion 1), then the sale at store A will be recommended to passenger 5.

[0038] Furthermore, Area DB91 records the following as a second area information: floor number "7th floor", area "Food Court B", type "Food & Beverage 2", content "-", and feature classification "N002". This indicates that Food Court B, which is classified as Food & Beverage 2, is located on the 7th floor. Although no content is registered for Food Court B, if there are, for example, discount coupons, this could be registered as "Content". This information can be entered from the store terminal 600 in Food Court B. The information announced in Figure 1 corresponds to feature classification "N002".

[0039] "Category" refers to classifications within restaurants, such as fast food, food courts (dining 2), and high-end Japanese restaurants. For example, if passenger 5 is a family with children, a family-friendly food court (dining 2) would be recommended.

[0040] Figure 4 is a diagram illustrating an example of the estimation result 92. The estimation result 92 is stored in the storage device 82. An identification ID is assigned to each image of passenger 5 (hereinafter also referred to as "passenger image"). The estimation result 92 records the passenger image and the characteristics of passenger 5 for each identification ID.

[0041] "Passenger image" is an image of passenger 5 taken by camera 19. "Passenger characteristics" are the characteristics of passenger 5 estimated from the passenger image. The characteristics of passenger 5 include at least one of the following: passenger 5's composition (information of companions), passenger 5's gender, passenger 5's age, passenger 5's clothing, and passenger 5's belongings (accessories).

[0042] The passenger composition (information about companions) indicates whether the passenger is traveling alone or in a group (family, friends, etc.). In addition, passenger characteristics are classified as "characteristic classifications" based on characteristics such as composition, gender, clothing, and belongings.

[0043] For example, in estimation result 92, corresponding to identification ID "ID001", the passenger image "Passenger Image G1", passenger characteristics such as composition "Family 1", gender "Female", age "40s", clothing "X1", possessions "Y1", and feature classification "N002" are recorded. In the example in Figure 1, passenger 5a is the passenger with identification ID "ID001".

[0044] In the estimation result 92, corresponding to the identification ID "ID002", the passenger image "Passenger Image G2", and the passenger characteristics, including composition "Family 1", gender "Male", age "Teens", clothing "X2", possessions "Y2", and feature classification "N002", are recorded. In the example in Figure 1, passenger 5b is the passenger with identification ID "ID002".

[0045] In this case, passengers 5a and 5b are determined to be from the same family (family 1). For example, in Figure 1, passengers 5a and 5b are standing close together, and in this case, server 80 determines that passengers 5a and 5b are a group. Furthermore, server 80 can determine that a woman in her 40s and a man in his teens are parent and child (family 1) based on other information (for example, that they look alike).

[0046] For example, if the configuration is Family 1 (in this case, parent and child), the feature classification "N002" is assigned. In Area DB91, Food Court B (Dining 2) on the 7th floor, which is suitable for family use, is registered as corresponding to feature classification "N002". Note that it is also possible to configure the system to assign multiple area information, such as restaurant information and event information, to a single feature classification.

[0047] In the estimation result 92, corresponding to the identification ID "ID003", the passenger image "Passenger Image G3", and the passenger characteristics, such as composition "Solo", gender "Female", age "30s", clothing "X3", possessions "Y3", and feature classification "N001", are recorded. In the example in Figure 1, passenger 5c is the passenger with identification ID "ID003".

[0048] For example, if clothing "X3" is high-end designer clothing, possessions "Y3" is a high-end designer bag, and the person is a single woman in her 30s or older, then the characteristic classification "N001" will be assigned. In area DB91, the sale information for store A on the 5th floor, which handles women's high-end brands (fashion 1), is registered as corresponding to such characteristic classification "N001". In this way, characteristic classifications can be assigned according to gender, age, clothing, and possessions. The assignment of characteristic classifications may be done manually by store employees, etc., or it may be configured to be assigned automatically using generation AI, etc.

[0049] Estimation result 92 records the following information corresponding to identification ID "ID004": passenger image "Passenger Image G4", passenger characteristics including composition "Family 2", gender "Male", age "50s", clothing "X4", possessions "Y4", and feature classification "N003".

[0050] Estimation result 92 records the following information corresponding to identification ID "ID005": passenger image "Passenger Image G5", passenger characteristics including composition "Family 2", gender "Female", age "50s", clothing "X5", possessions "Y5", and feature classification "N003".

[0051] For example, if a couple is elderly (family 2), they will be assigned the characteristic classification "N003". For example, in area DB91, a calm Japanese restaurant suitable for elderly couples may be registered as corresponding to the characteristic classification "N003".

[0052] Figure 5 is a diagram illustrating an example of area information display. As mentioned above, passenger 5c, who is waiting for elevator car 10 at the boarding area, is passenger with identification ID "ID003". Passenger 5c is presented with event with feature classification "N002" (sale information at store A on the 5th floor).

[0053] The user terminal 700 has dedicated application software (apps) pre-installed. These apps include, for example, apps that display advertisements in shopping centers (commercial facilities) and apps that provide route guidance within shopping centers.

[0054] Server 80 estimates the passenger's characteristics based on the image of passenger 5c (passenger image G3) captured by camera 19. Then, it extracts sale information for store A on the 5th floor as area information that matches the estimated characteristic classification "N001". This information is transmitted to passenger 5c's user terminal 700.

[0055] As a result, the message "There is a sale at store A on the 5th floor. Shall we go to the 5th floor?" is displayed on the app on passenger 5c's user terminal 700. Alternatively, a QR code may be displayed on a display installed inside the basket 10. By passenger 5c scanning this QR code, they can access the server 80 and display the above message on the user terminal 700.

[0056] If passenger 5c clicks the "Yes" button (indicates their intention), the destination floor (car call) of the 5th floor is automatically registered in the car 10 in which passenger 5c is riding. As a result, car 10 responds to the call for the 5th floor and stops and opens its doors at the 5th floor. If passenger 5c clicks the "No" button, the destination floor of the 5th floor is not registered.

[0057] The following explanation will use a flowchart. Figure 6 is a flowchart of the processes performed by the utilization promotion system 100. Hereafter, "step" will also be simply referred to as "S". This process is performed by the control device 20, the server 80, and the user terminal 700. Each process should be configured to repeat periodically.

[0058] In this embodiment, the server 80 determines area information for an area that it recommends passenger 5 to use (hereinafter also referred to as the "recommended area") from among area information for multiple areas, based on the image of passenger 5 (passenger image) taken by the camera 19. Specifically, the server 80 estimates the characteristics of passenger 5 based on the image of passenger 5. Then, the server 80 determines the area information for the recommended area according to the characteristics of passenger 5.

[0059] The user terminal 700, announcement device 72, and information device 73, which serve as notification devices, notify passengers of recommended area information by voice or display. When the server 80 determines that passenger 5 has indicated their intention to use the recommended area after being notified of the recommended area information by the notification device, it sends a command to the control device 20 of elevator 1 to register a call for the floor where the recommended area is located in elevator car 10.

[0060] Specifically, when the announcement device 72 makes an announcement (notification) or the information device 73 displays (notification), the server 80 determines whether or not passenger 5 has raised their hand based on the image of passenger 5 captured by the camera 19. If the server 80 determines that passenger 5 has raised their hand, it determines that passenger 5 has indicated their intention to use the recommended area. In addition, as described above, it is also possible to determine whether or not the above indication has been made based on the voice spoken by passenger 5. The announcement device 72 announces area information for the recommended area.

[0061] Furthermore, server 80 transmits area information for the recommended area to user terminal 700. When user terminal 700 receives area information for the recommended area from server 80, it displays (notifies) the area information for the recommended area. Server 80 determines that passenger 5 has expressed their intention to use the recommended area when user terminal 700 performs an operation to agree to use the recommended area (clicks "Yes"). The following is an explanation following the flowchart.

[0062] When this process begins, in S21, the control device 20 acquires images (referred to as "camera images") from multiple cameras 19 installed in the landing and inside the elevator car 10, and transmits the camera images to the server 80.

[0063] In S11, when the server 80 acquires a camera image from the control device 20, it extracts passenger images from the camera image. For example, in the example in Figure 1, the server 80 extracts passenger image G1 of passenger 5a and passenger image G2 of passenger 5b from the camera image acquired from the camera 19 inside the elevator car 10 (see Figure 4). Passenger image G1 is an image cropped from the camera image showing the entire body of passenger 5a. Passenger image G2 is an image cropped from the camera image showing the entire body of passenger 5b. The extraction of passenger images can be carried out using known techniques. Similarly, the server 80 extracts passenger image G3 of passenger 5c from the camera image acquired from the camera 19 at the boarding area (see Figure 4).

[0064] In S12, server 80 retrieves area information from area DB 91 (Figure 3). For example, area information with feature classification "N001" is retrieved, which includes floor "5th floor", area "Store A", type "Fashion 1", and content "Sale". Additionally, area information with feature classification "N002" is retrieved, which includes floor "7th floor", area "Food Court B", and type "Food & Drink 2".

[0065] In S13, server 80 determines passenger characteristics from passenger images and further determines area information corresponding to the characteristic classification. The results are recorded in estimation result 92 (Figure 4).

[0066] For example, server 80 assigns identification ID "ID001" to passenger 5a in passenger image "Passenger Image G1", determines the passenger's characteristics as follows: composition "Family 1", gender "Female", age "40s", clothing "X1", possessions "Y1", and determines the characteristic classification "N002".

[0067] Furthermore, server 80 assigns identification ID "ID002" to passenger 5b in passenger image "Passenger Image G2", determines the passenger's characteristics as follows: composition "Family 1", gender "Male", age "Teens", clothing "X2", and possessions "Y2", and determines the characteristic classification "N002".

[0068] Furthermore, server 80 assigns identification ID "ID003" to passenger 5c in passenger image "Passenger Image G3", determines the passenger's characteristics as follows: composition "solo", gender "female", age "30s", clothing "X3", possessions "Y3", and determines the characteristic classification as "N001".

[0069] The method for determining passenger characteristics from passenger images can be, for example, the method shown in the second embodiment (the method of estimation using AI). Next, the server 80 searches the area DB 91 using the feature classification as a search key and determines the area information. For example, for passenger 5c, the area DB 91 is searched using the feature classification "N001" and the area information that matches it (sale information for store A on the 5th floor) is determined.

[0070] In S14, the server 80 transmits area information corresponding to the determined feature classification to the user terminal 700 and the control device 20. For example, in this example, for passengers 5a and 5b, who are determined to be "Family 1", the area information for the determined feature classification "N002" (floor "7th floor", area "Food Court B", type "Dining 2", content "-") is transmitted. Also, for passenger 5c, who is determined to be "Alone", the area information for the determined feature classification "N001" (floor "5th floor", area "Store A", type "Fashion 1", content "Sale") is transmitted.

[0071] In S22, the control device 20 causes the announcement device 72 to notify area information based on an announcement command, and also causes the information device 73 to display area information based on a display command. As a result, the announcement device 72 makes an announcement based on the announcement command. For example, the announcement device 72 in the car 10 on which passengers 5a and 5b are riding outputs the announcement "Are you going to the 7th floor food court B?" based on the area information of feature classification "N002". The information device 73 also displays information based on a display command. For example, the information device 73 displays "Are you going to the 7th floor food court B?".

[0072] Furthermore, passenger 5c is at the boarding area. The announcement device 72 on the boarding area, based on the area information of feature classification "N001", outputs the announcement "There is a sale at store A on the 5th floor. Are you going to the 5th floor?" (or "Are you going to the sale at store A on the 5th floor?"), and also displays the same content on the information device 73. When passenger 5c boards the boarding area, the server 80 determines, based on the camera images, that passenger 5 detected by the boarding area camera 19 and passenger 5 detected by the boarding area camera 19 are the same person, and determines that passenger 5 has boarded the boarding area. If an indication of intent is made at the boarding area, the destination floor of the boarding area will be registered. An indication of intent may also be made inside the boarding area based on the guidance provided in the boarding area.

[0073] In S23, the control device 20 transmits the camera image from camera 19 and the audio of passenger 5 collected from the microphone during the announcement in S22 to the server 80. In S15, the server 80 makes a judgment about the passenger's intentions based on the camera image and audio received from the control device 20. For example, in the example shown in Figure 1, an announcement corresponding to the determined feature classification "N002" (Are you going to food court B on the 7th floor?) is made to passengers 5a and 5b, who have been determined to be "Family 1".

[0074] In response, passenger 5a, who is the target of the announcement for feature classification "N002," raises their hand. The control device 20 determines, based on the camera image, that passenger 5a, who is the target of feature classification "N002," has raised their hand, and determines that an expression of intent regarding feature classification "N002" has been made. Whether or not a hand has been raised can be determined by image judgment using publicly available technology. Even if it is determined that passenger 5b, who is the target of feature classification "N002," has raised their hand, it is determined that an expression of intent regarding feature classification "N002" has been made. However, even if other passengers raise their hands, it is not determined that an expression of intent has been made.

[0075] Furthermore, when passenger 5c is detected to have boarded car 10, an announcement corresponding to feature classification "N001" is made (There is a sale at store A on the 5th floor. Are you going to the 5th floor?). In this case, if it is determined that passenger 5c, who is subject to feature classification "N001," has raised their hand, it is determined that an expression of intent regarding feature classification "N001" has been made. Note that an expression of intent may also be determined by any action other than raising a hand, such as waving.

[0076] In this embodiment, announcements are made not only by the announcement device 72 but also by the information device 73 (display device). In this case, the message "Are you going to Food Court B on the 7th floor?" is displayed on the display device to passengers 5a and 5b. In addition, passengers can express their intentions not only by raising their hands but also by speaking. For example, microphones are installed in the landing and inside the elevator car 10. The server acquires sound from the microphones and analyzes the content of the speech using known technology. The server 80 determines that an expression of intent has been made when it determines that an expression such as "Yes," "I'm going," or "On the 7th floor" has been made. It is also possible to configure the system so that announcements are not made by the announcement device 72 or the information device 73 in the landing, and announcements are made only inside the elevator car 10. Furthermore, the information device 73 and microphones do not need to be installed.

[0077] In S31, the user terminal 700 displays area information to the user terminal 700 that is subject to feature classification. As shown in the example in Figure 5, the user terminal 700 of passenger 5c, who is subject to feature classification "N001", displays "There is a sale at store A on the 5th floor. Shall we go to the 5th floor?". In addition, the user terminal 700 of passenger 5a, who is subject to feature classification "N002", displays "Shall we go to food court B on the 7th floor?".

[0078] In S32, the user terminal 700 sends the declaration of intent information to the server 80 and terminates processing on the user terminal 700. For example, in the example in Figure 5, if passenger 5c clicks "Yes" on the user terminal 700, declaration of intent information indicating that a declaration of intent has been made is sent to the server 80. On the other hand, if passenger 5c clicks "No" on the user terminal 700, declaration of intent information indicating that no declaration of intent has been made is sent to the server 80.

[0079] Even if "Yes" is not explicitly clicked, it may be determined that an expression of intent has been made through other actions. For example, if a link to area information is displayed on the user terminal 700, and passenger 5 clicks that link to display the specific content of the area information, it may be determined that an expression of intent has been made.

[0080] In S16, Server 80 determines whether or not there is an expression of intent. If Server 80 determines that there is no expression of intent (NO in S16), Server 80 terminates its processing. On the other hand, if Server 80 determines that there is an expression of intent (YES in S16), it requests Control Device 20 to register the destination floor corresponding to the area information, and then terminates Server 80's processing.

[0081] If server 80 receives information from user terminal 700 indicating that no intention was expressed, or if it determines in S15 that no intention was expressed based on camera images and audio, it will determine NO in S16. On the other hand, if server 80 receives information from user terminal 700 indicating that an intention was expressed, or if it determines in S15 that an intention was expressed based on camera images and audio, it will determine YES in S16.

[0082] In S24, the control device 20 registers the destination floor as requested and terminates its processing. For example, if a passenger expresses their intention for feature classification "N001", the destination floor to the 5th floor is registered (Figure 5). If a passenger expresses their intention for feature classification "N002", the destination floor to the 7th floor is registered (Figure 1). If a passenger expresses their intention at the boarding area, the destination floor is registered in the elevator car 10 when the passenger boards the elevator car 10. As a result, the passenger 5 who expressed their intention can get off at the floor where the store or other establishment they expressed their intention for is located, without having to press the elevator car call button (destination floor call button).

[0083] Furthermore, it is also possible to set time zones in the feature classification of Area DB91. For example, it may be set so that restaurants suitable for lunch are extracted during lunchtime. Similarly, it may be set so that restaurants suitable for dinner are extracted during dinnertime.

[0084] Alternatively, instead of using feature classification, a conversion table or conversion rule may be prepared to associate "passenger characteristics" with "area information." For example, if a passenger's characteristics include the composition "Family 1," a conversion table or conversion rule may be prepared to extract area information that includes the category "Dining 2." Or, if a passenger is a single woman in her 30s or older and has certain characteristics (luxury brands) in her clothing and belongings, a conversion table or conversion rule may be prepared to extract area information that includes the category "Fashion 1."

[0085] Additionally, a "number of people" field could be included as a passenger characteristic. For example, if there is a large group, restaurants that can accommodate a large number of people could be recommended. Furthermore, a "body temperature" field could be included as a passenger characteristic. For example, if a passenger has a high body temperature, facilities or restaurants that can cool them down could be recommended.

[0086] The utilization promotion system 100 is a system that promotes the use of multiple areas (shops, event venues) provided in a building (for example, a commercial facility). The utilization promotion system 100 comprises a camera 19, a server 80, and a notification device (user terminal 700, announcement device 72). The camera 19 photographs passengers 5 inside the elevator car 10 of the elevator 1 installed in the building. Based on the image of passengers 5 taken by the camera 19, the server 80 determines the area information of a recommended area to be recommended for use by passengers 5 from among the area information of multiple areas. The notification device notifies the area information of the recommended area. When the server 80 determines that passengers 5 have indicated their intention to use the recommended area after the area information of the recommended area has been notified by the notification device, it sends a command to the control device 20 of the elevator 1 to register a car call for the floor where the recommended area is located in the elevator car 10.

[0087] In this way, area information for a recommended area determined based on the image of passenger 5 captured by camera 19 is notified, and if passenger 5 indicates their preference, a call for the elevator car on the floor where the recommended area is located is automatically registered. As a result, passenger 5 can disembark on the floor with the recommended area simply by indicating their preference, without having to press the elevator car call button. In this way, firstly, by providing information on recommended areas within the building tailored to each individual passenger 5, it contributes to improving the value of the elevator equipment. Secondly, for tenants such as shops located in the building, targeted advertising to passenger 5 in the elevator can be delivered, improving the effectiveness of the tenants' advertising. Thirdly, for building users (passengers 5), there are more opportunities to obtain information on shops and other facilities within the building on an individual basis, and receiving this information can lead to increased use and purchase intent of services, as well as improved convenience and satisfaction. Furthermore, for users, personalized suggestions are automatically provided without having to set their preferences or information on stores they wish to use in advance on the app. For building owners, stimulating user purchasing intent and improving convenience leads to increased foot traffic to the building, which in turn leads to higher sales for individual stores and an overall increase in the building's value. In this way, information obtained from cameras installed in elevators can be used to promote the use of the building.

[0088] Server 80 includes a storage device 82 that stores area information for multiple areas. Each of the area information entries includes discount information and event information. This allows passengers 5 to go to their desired area based on the discount information and event information.

[0089] Server 80 estimates the characteristics of passenger 5 based on an image of passenger 5. The characteristics of passenger 5 include at least one of the following: gender, age, clothing, belongings, and information about companions. Server 80 determines the area information of the recommended area according to the characteristics of passenger 5. As a result, recommended areas are selected according to the characteristics of passenger 5, such as gender, age, clothing, belongings, and companions (e.g., family), so areas that are more tailored to passenger 5's preferences can be presented.

[0090] The notification device is an announcement device 72 installed inside the elevator car 10. The announcement device 72 announces area information for the recommended area. The server 80 determines whether or not passenger 5 raised their hand based on the image of passenger 5 taken by the camera 19 when the announcement is made. If the server 80 determines that passenger 5 raised their hand, it determines that passenger 5 has indicated their intention to use the recommended area. As a result, passenger 5 can disembark at the floor where the recommended area is located simply by raising their hand, without having to press the elevator car call button, thus improving convenience for passenger 5.

[0091] The notification device is a user terminal 700, carried by passenger 5, that can communicate with server 80. Server 80 transmits area information of the recommended area to user terminal 700. When user terminal 700 receives area information of the recommended area from server 80, it displays the area information of the recommended area. Server 80 determines that passenger 5 has expressed their intention to use the recommended area when an operation indicating consent to use the recommended area is performed on user terminal 700. As a result, passenger 5 can disembark at the floor where the recommended area is located simply by agreeing to use it when the area information is displayed on user terminal 700, without having to press the elevator call button, thus improving convenience for passenger 5.

[0092] [Second Embodiment] In the second embodiment, the utilization promotion system 100 is configured to estimate passenger characteristics and area information from passenger images using AI (Artificial Intelligence). Specifically, the server 80 infers passenger characteristics and area information from passenger images using a trained model. The trained model is a model for inferring passenger characteristics and area information from passenger images. The following describes the differences from the utilization promotion system 100 according to the first embodiment, and omits the description of the parts that are the same as the utilization promotion system 100 according to the first embodiment.

[0093] Figure 7 shows an example of training data 93 according to the second embodiment. Training data 93 is data that represents the training dataset used for training.

[0094] The input data for training data 93 includes passenger images. The output data for training data 93 is the output result given the input data. The output result includes passenger characteristics and area information. In training data 93, the output result is used as ground truth data (training data). The input and output data for training data 93 are the same as those for area DB 91 and estimation result 92.

[0095] For example, suppose passenger images G1 and G2 are input data for training data set number 1. For passenger image G1, passenger features R1 and area information N1 are set as output data (ground truth data) for the input data, and these are prepared as training dataset set number 1. For passenger image G2, passenger features R2 and area information N1 are set as output data (ground truth data) for the input data, and these are prepared as training dataset set number 1.

[0096] For example, for passenger image G1, the following characteristics are given as ground truth data: composition "family", gender "female", age "40s", clothing "X1", and possessions "Y1". Also, for passenger image G1, the following characteristics are given as ground truth data: composition "family", gender "male", age "teens", clothing "X2", and possessions "Y2". Furthermore, for passenger images G1 and G2, the following area information is given as common ground truth data: floor "7th floor", area "food court B", type "food and drink 2", and content "-".

[0097] Furthermore, let's assume that passenger image G3 is input data as training data set number 2. In this case, passenger features R3 and area information N2 are set as output data (ground truth data) for the input data of passenger image G3, and these are prepared as training datasets. For example, for passenger image G3, the ground truth data is given as passenger features: composition "solo", gender "female", age "30s", clothing "X3", possessions "Y3", and area information: floor "5th floor", area "Store A", type "fashion 1", content "sale".

[0098] Figure 8 shows an example configuration of the learning device 201. In the second embodiment, the utilization promotion system 100 includes the learning device 201. The functions implemented by the learning device 201 may also be implemented by the server 80.

[0099] The learning device 201 includes a data acquisition unit 202 and a model generation unit 203. The functions of the data acquisition unit 202 and the model generation unit 203 are realized by the processor executing a program stored in memory.

[0100] The processor acquires training data, including passenger images, passenger characteristics, and area information, and uses this training data to generate a trained model that infers passenger characteristics and area information (output data) from passenger images (input data).

[0101] The data acquisition unit 202 acquires input data and output data (ground truth data (training data)) as training data. The model generation unit 203 generates a trained model that infers output data from input data based on the training data consisting of the combination of input data and output data output from the data acquisition unit 202. The model generation unit 203 can perform training using, for example, supervised learning using a neural network.

[0102] The model generation unit 203 can also use deep learning, which learns to extract features themselves, as its learning algorithm. The model generation unit 203 stores the trained model generated by performing the learning process in the trained model memory device 300.

[0103] Figure 9 is a flowchart showing the learning process of the learning device 201. In S501, the data acquisition unit 202 acquires input data and output data (training data).

[0104] In S502, the model generation unit 203 generates a trained model that infers output data from input data using so-called supervised learning, according to training data consisting of a combination of input data and output data (training data) acquired by the data acquisition unit 202.

[0105] In other words, as shown in Figure 8, the trained model, when inputting passenger images, infers passenger characteristics and area information as output data.

[0106] In S503, the trained model storage device 300 stores the trained model generated by the model generation unit 203.

[0107] Figure 10 shows an example of the configuration of the inference processing unit 401. In the utilization promotion system 100 according to the second embodiment, the server 80 is equipped with the inference processing unit 401.

[0108] The inference processing unit 401 includes a data acquisition unit 402 and an inference unit 403. The functions of the data acquisition unit 402 and the inference unit 403 are realized, for example, by the processing unit 81 of the server 80 executing a program stored in the storage device 82.

[0109] Server 80 infers passenger characteristics and area information from passenger images using a pre-trained model for inferring passenger characteristics and area information from passenger images. Data acquisition unit 402 acquires passenger images as input data.

[0110] The inference unit 403 uses the trained model to infer passenger characteristics and area information as output data. In other words, the inference unit 403 can output output data inferred from the input data by inputting the input data acquired by the data acquisition unit 402 into the trained model.

[0111] Figure 11 is a flowchart showing the inference procedure performed by the inference processing unit 401. In S601, the data acquisition unit 402 acquires input data. In S602, the inference unit 403 inputs the input data to the trained model stored in the trained model memory device 300 and obtains output data. In S603, the inference unit 403 outputs the output data obtained by the trained model, i.e., passenger characteristics and area information.

[0112] In the first embodiment, in S13, passenger characteristics and area information are determined from the passenger image. In the second embodiment, passenger characteristics and area information are determined from the passenger image using the method described above.

[0113] As explained above, the trained model is a model for inferring passenger features and area information from passenger images. Server 80 infers passenger features and area information from passenger images. This allows for more accurate acquisition of passenger features and area information using the trained model.

[0114] In the second embodiment, the inference processing by the inference unit 403 may be performed on an external server. The external server is, for example, a web server equipped with an inference processing unit which is artificial intelligence. For example, the inference unit 403 in Figure 10 sends passenger images (input information) acquired from the data acquisition unit 402 to the external server. The passenger images are input to the inference processing unit which is artificial intelligence on the external server, and passenger characteristics and area information are obtained as output information. For example, when the inference unit 403 receives passenger images as input, it sends a request to the external server via an API (Application Programming Interface) to obtain passenger characteristics and area information as output information. The inference unit 403 acquires passenger characteristics and area information from the external server and outputs passenger characteristics and area information as estimation results.

[0115] The artificial intelligence described above may be a generative AI (Generative Artificial Intelligence) that includes large language models (LLMs) and diffusion models. The generative AI may also be a multimodal AI that handles input and output of text, images, audio, and video, such as GPT and Gemini. Furthermore, the generative AI may be configured to undergo additional training through fine-tuning to improve the accuracy of the output (passenger characteristics and area information) in relation to the input (passenger images). Alternatively, Retrieval-Augmented Generation (RAG) may be applied to improve output accuracy by searching a database of past maintenance information accumulated by the maintenance company that manages the elevators, and then inputting the search results, along with other input information, into the generative AI.

[0116] By configuring the system as described above, the second embodiment also makes it possible to promote the use of the building by utilizing information obtained from a camera installed in the elevator.

[0117] Furthermore, the system may be configured to train a pre-trained model so that when a passenger image is input, it outputs the characteristics of the passenger. Also, as in the first embodiment, the "Feature Classification" item may be included for both the passenger characteristics and the area information.

[0118] Furthermore, the AI ​​can be used to output the reason for selecting area information based on the passenger's characteristics. For example, for a single female with identification ID 003, the reason for recommending store A in Fashion 1 could be an announcement or display such as, "Your clothing and bag are from luxury brands, so we recommend store A, which carries similar luxury brands."

[0119] Furthermore, the store terminal 600 may be configured to display the estimated result 92. This allows the store to receive feedback on the actual behavior of passenger 5. In addition, the store terminal 600 may be configured to also display the reason for selecting the area information as described above. This allows the store to utilize this information and helps improve the accuracy of future area information estimations.

[0120] [Note] The embodiments described above are specific examples of the following appendix.

[0121] (Note 1) A utilization promotion system that facilitates the use of multiple areas within a building, A camera installed in the aforementioned building to photograph passengers inside the elevator car, A server that determines, based on the image of the passenger captured by the camera, the area information of a recommended area that is recommended for use by the passenger from among the area information of the multiple areas, The system includes a notification device that notifies area information of the aforementioned recommended area, A usage promotion system in which, when the server is notified of the area information of the recommended area by the notification device, and the server determines that the passenger has indicated their intention to use the recommended area, the server transmits a command to the elevator control device to register a call for the floor on which the recommended area is located in the elevator car.

[0122] (Note 2) The server includes a storage device that stores area information for the plurality of areas, Each of the area information for the aforementioned multiple areas includes discount information and event information, as described in Appendix 1, for the utilization promotion system.

[0123] (Note 3) The server estimates the passenger's characteristics based on the passenger's image, The passenger's characteristics include at least one of the following: gender, age, clothing, belongings, and information about accompanying persons. The server determines area information for the recommended area according to the characteristics of the passenger, as described in Appendix 1 or Appendix 2, which is a usage promotion system.

[0124] (Note 4) The notification device is an announcement device installed inside the elevator car. The announcement device announces area information for the recommended area. The aforementioned server, Based on the image of the passenger taken by the camera during the announcement, it is determined whether or not the passenger raised their hand. A usage promotion system according to any one of the appendices 1 to 3, which determines that the passenger has expressed an intention to use the recommended area when it is determined that the passenger has raised their hand.

[0125] (Note 5) The notification device is a user terminal possessed by the passenger that is capable of communicating with the server. The server transmits area information of the recommended area to the user terminal. When the user terminal receives area information for the recommended area from the server, it displays the area information for the recommended area. The aforementioned server determines that the passenger has expressed their intention to use the recommended area when an operation indicating consent to use the recommended area is performed on the user terminal, as described in any of the appendices 1 to 3.

[0126] (Note 6) A control method for a utilization promotion system that facilitates the use of multiple areas provided in a building, The aforementioned utilization promotion system comprises a camera for photographing passengers inside the elevator car installed in the building, a server, and a notification device. The control method described above is Based on the image of the passenger captured by the camera, the server determines the area information of a recommended area that it recommends the passenger use from among the area information of the multiple areas. The steps include: the notification device notifying area information of the recommended area; A control method comprising the steps of: when the notification device notifies the passenger of area information for the recommended area, the server determines that the passenger has indicated their intention to use the recommended area, and the server transmits a command to the elevator control device to register a call for the floor on which the recommended area is located in the elevator car.

[0127] The embodiments disclosed herein are illustrative and not limited to those described herein. The scope of the present invention is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0128] 1 Elevator, 5,5a~5c Passengers, 10 Car, 19 Camera, 20 Control device, 21 Processing device, 22 Storage device, 30 Car device, 40 Landing device, 50 Communication device, 72 Announcement device, 73 Information device, 80 Server, 81 Processing device, 82 Storage device, 91 Area DB, 92 Estimation results, 93 Training data, 100 Usage promotion system, 201 Learning device, 202 Data acquisition unit, 203 Model generation unit, 300 Trained model storage device, 401 Inference processing unit, 402 Data acquisition unit, 403 Inference unit, 600 Store terminal, 700 User terminal.

Claims

1. A utilization promotion system that facilitates the use of multiple areas within a building, A camera installed in the aforementioned building to photograph passengers inside the elevator car, A server that determines, based on the image of the passenger captured by the camera, the area information of a recommended area that is recommended for use by the passenger from among the area information of the multiple areas, The system includes a notification device that notifies area information of the aforementioned recommended area, A usage promotion system in which, when the server is notified of the area information of the recommended area by the notification device, and the server determines that the passenger has indicated their intention to use the recommended area, the server transmits a command to the elevator control device to register a call for the floor on which the recommended area is located in the elevator car.

2. The server includes a storage device that stores area information for the plurality of areas, The usage promotion system according to claim 1, wherein each of the area information for the plurality of areas includes discount information and event information.

3. The server estimates the passenger's characteristics based on the passenger's image, The passenger's characteristics include at least one of the following: gender, age, clothing, belongings, and information about accompanying persons. The user promotion system according to claim 1, wherein the server determines area information for the recommended area according to the characteristics of the passenger.

4. The notification device is an announcement device installed inside the elevator car. The announcement device announces area information for the recommended area. The aforementioned server, Based on the image of the passenger taken by the camera during the announcement, it is determined whether or not the passenger raised their hand. The usage promotion system according to any one of claims 1 to 3, wherein it is determined that the passenger has raised their hand, and the system determines that the passenger has expressed their intention to use the recommended area.

5. The notification device is a user terminal possessed by the passenger that is capable of communicating with the server. The server transmits area information of the recommended area to the user terminal. When the user terminal receives area information for the recommended area from the server, it displays the area information for the recommended area. The usage promotion system according to any one of claims 1 to 3, wherein the server determines that the passenger has expressed their intention to use the recommended area when an operation indicating consent to use the recommended area is performed on the user terminal.

6. A control method for a utilization promotion system that facilitates the use of multiple areas provided in a building, The aforementioned utilization promotion system comprises a camera for photographing passengers inside the elevator car installed in the building, a server, and a notification device. The control method described above is Based on the image of the passenger captured by the camera, the server determines the area information of a recommended area that it recommends the passenger use from among the area information of the multiple areas. The steps include: the notification device notifying area information of the recommended area; A control method comprising the steps of: when the notification device notifies the passenger of area information for the recommended area, the server determines that the passenger has indicated their intention to use the recommended area, and the server transmits a command to the elevator control device to register a call for the floor on which the recommended area is located in the elevator car.

Images