Station yard maintenance system, station yard maintenance method and station yard maintenance program

Abstract

To deal with a body present in a station yard from a system side.SOLUTION: A station yard maintenance system 1 comprises an image recognition part 22a, a command generation part 22c and a robot 30. The image recognition part 22a acquires an image generated by capturing the inside of a station yard by an imaging device 11 and specifies the category and location of a body included in the image through image recognition. The command generation part 22c generates a command to deal with the body based upon the category and location of the body that the image recognition part 22a specifies. The robot 30 receives the command that the command generation part 22c generates and moves to the location of the body according to the command to deal with the body.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present application primarily relates to a system for dealing with objects in a station premises and maintaining the environment within the station premises. [Background technology]

[0002] Patent Document 1 discloses a manners violation detection system. This system includes a sensor that detects manners violations, a camera that captures images of the station premises, and a station work remote monitoring system. The station work remote monitoring system is a system in which station staff monitor the station from a remote location. When the sensor detects a manners violation, the sensor transmits an alarm signal to the monitoring system. This allows station staff of the station work remote monitoring system to know that a manners violation has occurred. In addition, the monitor system display shows an image obtained from a camera close to the sensor that detected the manners violation. This allows station staff to know the details of the manners violation. Patent Document 1 lists smoking, littering, etc. as examples of manners violations. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-232871 A Summary of the Invention [Problem to be solved by the invention]

[0004] The etiquette violation detection system of Patent Document 1 can detect etiquette violations, but cannot deal with them. Station staff must deal with etiquette violations. However, station staff have many tasks to perform, and may not be able to spare the time to deal with etiquette violations. Furthermore, this issue is not limited to dealing with etiquette violations, but is a common issue for all work that station staff do within the station.

[0005] The present application has been made in consideration of the above circumstances, and its main objective is to provide an in-station maintenance system that is capable of taking necessary measures against objects present within a station. [Means for solving the problem]

[0006] The problem to be solved by the present application is as described above. Next, the means for solving this problem and the effects thereof will be described.

[0007] According to a first aspect of the present application, there is provided an in-station maintenance system having the following configuration. That is, the in-station maintenance system includes an image recognition unit, a command generation unit, and a robot. The image recognition unit acquires an image generated by capturing an image of the inside of a station with an imaging device, and identifies a category and a position of an object shown in the image by image recognition. The command generation unit generates a command for dealing with the object based on the category and position of the object identified by the image recognition unit. The robot receives the command generated by the command generation unit, moves to the position of the object based on the command, and deals with the object.

[0008] According to a second aspect of the present application, there is provided the following in-station maintenance method. That is, in the in-station maintenance method, an image generated by capturing an image of the inside of a station with an imaging device is acquired, and a category and a position of an object shown in the image are identified by image recognition. In the in-station maintenance method, a command for dealing with the object is generated based on the category and position of the object identified by the image recognition, and transmitted to a robot. In the in-station maintenance method, the robot that has received the command is moved to the position of the object, and the robot is made to deal with the object.

[0009] According to a third aspect of the present application, there is provided a station premises maintenance program for causing a computer to execute the following processes. That is, the station premises maintenance program includes a process of acquiring an image generated by capturing an image of the station premises with an imaging device, and identifying a category and a position of an object captured in the image by image recognition. The station premises maintenance program includes a process of generating a command for dealing with the object based on the category and position of the object identified by the image recognition, and transmitting the command to a robot. Effect of the Invention

[0010] According to the present application, the system can take necessary measures against objects present within the station premises. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram of a station maintenance system and related devices according to an embodiment of the present application; [Diagram 2] 13 is a flowchart of a process performed by a processing server. [Diagram 3] 11 is a flowchart showing a process for identifying an object category and location. [Figure 4] 11 is a flowchart illustrating an example of identifying an object category and location. [Diagram 5] FIG. 13 is a diagram showing a countermeasure information database in which object categories are associated with countermeasure candidates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Next, an embodiment of the present application will be described with reference to the drawings.

[0013] The station premises maintenance system 1 is a system that reduces the burden on station staff by performing some of the work that station staff currently perform on their behalf. The station premises maintenance system 1 is a system for performing an in-station maintenance method. The in-station maintenance method of this embodiment is a method of acquiring information such as images of the station premises, identifying necessary measures based on the acquired information, and performing the necessary measures using a robot.

[0014] FIG. 1 shows an imaging device 11, a sensor 12, and a station information DB 13 as components for providing information within a station.

[0015] The imaging device 11 captures images of the station premises, generates images, and stores them. The images generated by the imaging device 11 are visible images obtained by converting visible light into images, and are not thermal images or distance images. The imaging device 11 may be fixed within the station premises, or may be movable or have a variable viewing angle. When the imaging device 11 is movable or has a variable viewing angle, it is preferable to store position information or orientation information together with the images. The imaging device 11 may be a camera whose main function is to capture still images, or a video camera whose main function is to capture videos. The imaging device 11 may also be a camera module incorporated in another device. Specifically, the imaging device 11 is a station premises camera, a robot camera, or a smartphone camera. The station premises camera is a camera fixed to a ceiling or a pillar in the station premises, and may be used as a security camera or the like. The robot camera is a camera module incorporated in a robot 30 described later. The smartphone camera is a camera module incorporated in a smartphone.

[0016] The imaging device 11 further has a communication function. The imaging device 11 transmits an image by wired communication or wireless communication. The image transmitted by the imaging device 11 is uploaded to the cloud via a PC or a router in the station. Alternatively, if the imaging device 11 is incorporated in a smartphone, the smartphone may upload the image to the cloud via a mobile network. It is preferable that the imaging device 11 transmits an image in association with information specifying the position or orientation of the image. For example, it is preferable to transmit an image in association with the above-mentioned position information or orientation information. Alternatively, if the identification information of the imaging device 11 and the position of the imaging device 11 are registered separately in association with each other, it is preferable to transmit an image in association with the identification information of the imaging device 11.

[0017] The sensor 12 detects information other than images of the station premises and stores the sensor information detected by the sensor. The manner in which the sensor 12 is provided, the information stored together with the sensor information, and the manner in which the sensor information is transmitted are the same as those of the imaging device 11. The sensor 12 is, for example, a heat sensor, a sound sensor, a three-dimensional sensor, or a vibration sensor. The heat sensor is an infrared camera or a temperature sensor, and stores thermal information such as a thermal image or an ambient temperature. The sound sensor is a microphone or the like, and stores sound information such as the content of surrounding sounds or the volume of sounds. The three-dimensional sensor is a radar, LiDAR, ultrasonic sensor, or the like, and stores three-dimensional detection information such as the three-dimensional shape of the surroundings. The vibration sensor stores the vibration level of surrounding objects as vibration information.

[0018] The station information DB13 is a database in which station information, which is various information related to a station, is registered, and is stored in a PC in the station or a server outside the station. The station information DB13 includes, for example, a station premises map or a lost property DB. The station premises map is divided into a plurality of areas, and each area is assigned a unique area number. The location of the station premises may be specified using the area number, or the location of the station premises may be specified using latitude and longitude information. The station premises map also includes information on various facilities installed in the station. The lost property DB is a database of lost property delivered to a station and lost property reported by passengers. Information such as the characteristics of the lost property, the date and time when it was found, the location where it was found, and the person who found it are registered in association with each lost property delivered to a station. Information such as the characteristics of the lost property, the date and time when it was found, the location where it was found, and the person who found it are registered in association with each lost property reported by passengers.

[0019] The station premises maintenance system 1 is provided with an image storage server 21 and a processing server 22 on the cloud. On the cloud refers to equipment constructed on a server or the like that is connected via the Internet and is located physically away from the station premises. Note that the image storage server 21 and the processing server 22 are not limited to being on the cloud, and may be provided in the station premises. Also, the image storage server 21 or the processing server 22 may be realized by one server device, or may be realized by a plurality of server devices working together.

[0020] The image storage server 21 stores images and the like transmitted by the imaging device 11. The image storage server 21 stores images and transmits them to the processing server 22. The image storage server 21 is not an essential component and can be omitted.

[0021] The processing server 22 generates a command to identify a necessary measure and transmit it to the robot based on information inside the station, specifically, images of the station, sensor information, station information, and public information. Here, the public information is information available on the Internet and is used to analyze information inside the station. The processing server 22 is a server device including a CPU, storage, memory, communication device, etc. The CPU executes a station maintenance program stored in the storage, and the processing server 22 realizes various functions. Specifically, the processing server 22 has functions as an image recognition unit 22a, an analysis unit 22b, and a command generation unit 22c. In this embodiment, one processing server 22 has the functions of the image recognition unit 22a, the analysis unit 22b, and the command generation unit 22c, but this is just an example, and a different server device may be used for each function.

[0022] The image recognition unit 22a performs image recognition on an image captured inside a station to identify the category and position of an object shown in the image. The category of an object is a name given to each classification when the object is classified from various viewpoints. In this embodiment, the object is classified according to the necessary measures. The object categories in this embodiment include lost property, objects fallen on the tracks, stolen goods, suspicious objects, garbage / dirt, and damaged facilities. Note that the object may be classified and categorized from a viewpoint different from that in this embodiment. For example, the object may be classified according to the size, weight, or importance of the object. Details of the image recognition will be described later.

[0023] The analysis unit 22b analyzes the sensor information. The analysis results of the sensor information are used to identify the object in more detail. For example, the analysis unit 22b can identify that the object is a cigarette, a suspicious chemical substance, or equipment that is generating abnormal heat based on the heat information. The analysis unit 22b can identify that an object has fallen onto the tracks or that the object is equipment that is generating an abnormal sound based on the sound information. The analysis unit 22b uses the three-dimensional detection information as auxiliary information for identifying the shape and size of the object. The analysis unit 22b uses the vibration information as auxiliary information for identifying whether the object is vibrating or not.

[0024] The analysis unit 22b may use the analysis results of the sensor information to identify the situation within the station premises. For example, when the analysis unit 22b analyzes the sound information and finds that it contains a passenger's scream, it determines that there may be a suspicious object or a stolen item. Furthermore, when the analysis unit 22b analyzes the sound information and finds that it contains the sound of rain, it determines that there may be a lost umbrella. Information other than sound information may also be used to identify the situation within the station premises. Note that the analysis unit 22b is not an essential component and may be omitted.

[0025] The command generating unit 22c generates a command for dealing with the object based on the category and position of the object identified by the image recognizing unit 22a and the analysis result of the analyzing unit 22b, and transmits the command to the robot 30. The process of the command generating unit 22c will be described in detail later.

[0026] The robot 30 is a general-purpose robot capable of autonomous travel and performing various tasks. The robot 30 includes a detection sensor 31, a control device 32, a work tool 33, a travel drive unit , a display unit 35, and a speaker .

[0027] The detection sensor 31 detects the surroundings of the robot 30. The detection sensor 31 is a camera, a radar, a LiDAR, or an ultrasonic sensor. By traveling based on the detection results of the detection sensor 31, the robot 30 does not collide with passengers or pillars. The detection sensor 31 is also used to identify the positional relationship with the object to be dealt with. The control device 32 is a microcomputer having a CPU, storage, memory, communication device, etc., and controls the work tool 33, the traveling drive unit 34, the display unit 35, and the speaker 36 based on received commands.

[0028] The work tool 33 is a tool for the robot 30 to perform work. The work tool 33 may be a dedicated tool for a specific work. An example of a dedicated tool is a tool for cleaning. The work tool 33 may also be a general-purpose tool that can be used for various works. An example of a general-purpose tool is a robot hand. The robot 30 may also be equipped with a tool changer. This allows the work tool 33 to be switched to handle various works. The control device 32 controls the work tool 33 based on the work included in the command. This allows the robot 30 to take the necessary measures.

[0029] The traveling drive unit 34 is a motor that drives the wheels, crawlers, legs, etc. to make the robot travel. The control device 32 controls the traveling drive unit 34 according to the target position that is also included in the command. This allows the robot 30 to move to the target position.

[0030] The display unit 35 is a liquid crystal display or an organic EL display, and is capable of displaying information. The control device 32 generates an image based on the information included in the command and displays it on the display unit 35. This makes it possible to notify information to passengers, station staff, etc.

[0031] The speaker 36 converts the electrical signal into sound and emits the sound. The control device 32 generates sound based on the information included in the command and outputs the sound from the speaker 36. This makes it possible to notify information to passengers, station staff, etc.

[0032] Next, the flow of processing performed by the station premises maintenance system 1 will be described with reference to Figs.

[0033] First, when the processing server 22 becomes ready for processing, it accesses the image storage server 21 and acquires images of the inside of the station (S101 in FIG. 2). In the station premises maintenance system 1, it is preferable to acquire images of various areas in order to identify objects in various areas and take measures against them. For example, it is preferable to acquire images for each area at a predetermined time interval.

[0034] The processing server 22, more specifically, the image recognition unit 22a, performs image recognition on the image of the station premises and identifies the category and position of the object (S102). More specifically, the image recognition unit 22a performs the process shown in the flowchart of Fig. 3 to identify the category and position of the object. Also, Fig. 4 shows a specific example of the process shown in Fig. 3.

[0035] First, the image recognition unit 22a specifies the range that an object occupies from an image of the station premises and extracts an image of the object (S201). For example, as shown in FIG. 4, the image recognition unit 22a extracts an image of an umbrella, which is a target object, from an image of a platform in the station premises. In addition, the image of the station premises includes multiple objects, and the image recognition unit 22a selects the target object by the following method. Here, it is assumed that the imaging device 11 is a camera fixed in the station premises. The image storage server 21 stores multiple images of the station premises captured by the same imaging device 11 in the past. Therefore, the image recognition unit 22a compares the image acquired this time with the past images, and preferentially identifies an object that is not captured in the past images and is captured in the image acquired this time. In the example of FIG. 4, since facilities such as benches and railroad cars are captured in the past images, the image recognition unit 22a identifies an umbrella, which is another object. Note that instead of the method of comparing with the past images, one or more sample images without objects may be prepared and the image acquired this time may be compared with the sample images.

[0036] In this embodiment, in step S101, the image acquired this time is compared with other images to identify an object that does not usually exist, in other words, an object that may require some kind of action. Note that the process of comparing the image acquired this time with other images is not essential. Instead of this process, for example, images of objects existing in a station premises from various angles may be stored in a database. Then, when the image recognition unit 22a acquires an object from an image, the image of the object stored in the database may not be extracted. In the example shown in FIG. 4, images of a railroad car and a bench are registered in advance, and images similar to the images of the railroad car and the bench are not extracted.

[0037] Next, the image recognition unit 22a identifies the name of the object based on the image of the object (S202). This process is performed using a known matching method, etc. For example, when an image of an umbrella is extracted as shown in Fig. 4, the image of the umbrella is matched with the database of the processing server 22 or an external database, and it is identified that the extracted image indicates "umbrella."

[0038] The process of step S102 may be performed using AI. For example, in the learning phase, machine learning is performed by associating images of various types or angles of objects present in the station with the names of the objects. This allows a judgment model to be constructed. In the judgment phase, images of objects extracted from images of the station premises are input to the judgment model. The judgment model outputs the name of the object based on the input image. Here, the judgment model learns the association between the image and the name of the object from various perspectives, so that the name of the object can be guessed even if the image is different from the image used for learning.

[0039] Next, the image recognition unit 22a identifies the position of the object within the station based on the position of the imaging device 11 that captured the image and the position of the object on the image (S203). The position of the imaging device 11 that captured the image is stored in association with the image, as described above. The position of the object on the image can be identified based on the positional relationship between the image and the object. Based on this information, the approximate position of the object can be identified. The position of the object is identified by the area number within the station, as described above. For example, the image may be divided into a grid pattern, and an area number may be assigned to each area. In the example shown in FIG. 4, an area number such as "1F, Area B, (2, 5)" is identified.

[0040] Since the image of the station premises is a two-dimensional image, it is difficult to accurately identify the three-dimensional position of an object. However, in this embodiment, the robot 30 ultimately identifies the position of the object, so it is sufficient to be able to identify the approximate position of the object.

[0041] The approximate size of the object can be identified based on the name of the object. If the size of the object can be identified, the distance between the imaging device 11 and the object can be identified based on the size of the object on the image and the imaging settings of the imaging device 11. This allows the position of the object within the station to be identified in more detail. Furthermore, if the same object appears in images captured by multiple imaging devices 11 at the same time, the amount of information for identifying the object's position increases, so the object's position can be identified in more detail. Furthermore, if the object shown in the image of the station can be associated with the object included in the three-dimensional detection information, the detailed position of the object within the station can be identified.

[0042] Next, the image recognition unit 22a checks the characteristics of the object against the lost-item DB to determine whether the identified object is registered in the lost-item DB (S204). The characteristics of the object include color, pattern, shape, etc. The lost-item DB here is a database of lost items that passengers have reported as having lost. If the characteristics of the object identified from the image match the characteristics of the object registered in the lost-item DB, the lost item that the passenger reported as having lost is found.

[0043] Next, the image recognition unit 22a identifies the category of the object based on the name, position, and movement of the object, and the result of matching with the lost property DB (S205). The category of the object is identified based on whether or not a predetermined condition is satisfied. For example, as shown in FIG. 4, when the result of matching with the lost property DB is a match, the image recognition unit 22a identifies the category of the object as "lost property". When the object is located on the railroad tracks, the image recognition unit 22a identifies the category of the object as "item that has fallen on the railroad tracks". When the name of the object is a bag, and the object is not moving and there is no person around, the image recognition unit 22a identifies the category of the object as "item that has fallen on the railroad tracks", more specifically, as an object that may be stolen. When the name of the object cannot be identified and the degree of heat generation, vibration, voices of passengers around, etc. is high, the image recognition unit 22a identifies the category of the object as "suspicious object". Furthermore, if the name of the object belongs to or is similar to garbage, dirt, etc., the image recognition unit 22a determines the category of the object as "garbage, dirt." Also, if the equipment in the station has an unusual shape, temperature, vibration, etc., the image recognition unit 22a identifies the category of the object as "damaged equipment."

[0044] The process of step S205 may be performed using AI. As described above, there is a clear correlation between the name of an object, the position of an object, and the sensor information, and the category of an object. Therefore, the category of an object can be estimated using machine learning. For example, in the learning phase, past cases in a station are used as learning data, and the name of an object, the position of an object, and the sensor information are associated with the category of an object, and machine learning is performed to construct a judgment model. In the judgment phase, the name of an object, the position of an object, and the sensor information are input to the judgment model. The judgment model outputs the category of an object based on the input data. Here, the judgment model learns the association between the various information and the category of an object from various perspectives, and therefore the category of an object can be flexibly and accurately estimated.

[0045] The learning data is not limited to past cases in the same station premises, and past cases in other station premises or public places may be used. This can improve the accuracy of determining the object category. In addition, if a new suspicious object occurs that the station premises maintenance system 1 has not been able to identify, images and sensor information of the new suspicious object may be additionally learned. This makes it possible to respond to the next time a similar suspicious object occurs.

[0046] By carrying out the above-mentioned processing, the image recognition unit 22a can identify the category and position of an object by performing image recognition on an image inside the station (S102).

[0047] As described above, the processing server 22 acquires images of the same area at a predetermined time interval. Then, the processing server 22 performs similar image recognition to identify the category and position of the object again. In addition, when identifying the category of the object, the processing server 22 also identifies the characteristics of the object, and therefore can identify the position change of the object including the characteristics, that is, the movement history of the identified object. Hereinafter, continuing to acquire the movement history of the identified object is referred to as "tracking the object." That is, the processing server 22 tracks the identified object based on images of the station premises generated over time (S103).

[0048] Next, the processing server 22 acquires sensor information within the station from the sensor 12 (S104). The processing server 22, more specifically, the analysis unit 22b, analyzes the sensor information as described above (S105).

[0049] Next, the processing server 22, specifically the command generation unit 22c, generates a command based on the category and position of the object, and the analysis result of the sensor information (S106). The command generation unit 22c transmits the generated command to the robot 30 to have it deal with the target object (S107). Furthermore, when the processing server 22 determines that the category of the object, the command transmitted to the robot 30, the progress of the deal, and the like satisfy the notification condition (S108), it notifies the station staff (S109). The notification to the station staff is, for example, by transmitting a message to an information device owned or managed by the station staff. The notification to the station staff may be performed by the robot 30 instead of by the processing server 22.

[0050] The command generated by the processing server 22 is, for example, a command to move the robot 30 to the position of an object. This command includes the position of the object. The control device 32 of the robot 30 drives the traveling drive unit 34 to move to the position indicated by the area number based on the position of the object included in the received command, specifically the above-mentioned area number. If the area number alone is not enough to get close to the object, the command may include the characteristics of the object. The characteristics of the object are the name, shape, color, etc. of the object. The control device 32 analyzes the detection result of the detection sensor 31 based on the object characteristics included in the command, and recognizes the target object. The control device 32 controls the traveling drive unit 34 to get close to the target object.

[0051] Furthermore, the commands generated by the processing server 22 include, for example, commands for the robot 30 to deal with an object. Based on the received command, the control device 32 controls the work tool 33, the traveling drive unit 34, the display unit 35, or the speaker 36 to deal with the object. The specific handling will be described below.

[0052] The command generator 22c generates a command for dealing with an object by using, for example, a handling information database shown in Fig. 5. The handling information database is a database in which handling candidates are associated with each object category.

[0053] In the handling information database, as handling options for the case where the category of the object is "lost item", the following are described: picking up the lost item by the robot 30, transporting the lost item to a storage location by the robot 30, notifying the station staff of the characteristics of the lost item, and registering the lost item in the lost item DB. Picking up refers to the robot 30 picking up the lost item using the work tool 33 and holding it. Transporting to the storage location refers to the robot 30 picking up the lost item, then moving to the storage location for the lost item, and handing the lost item over to the storage location. Notifying the station staff of the characteristics of the lost item refers to notifying the station staff of the characteristics of the lost item. Also, instead of notifying the station staff of the characteristics of all the identified lost items, the notification may be made only when a predetermined condition, i.e., a notification condition, is satisfied. For example, the notification condition may be that the object identified by the processing server 22 has already been registered in the lost item DB. This allows the station staff to be immediately notified that the lost item that the passenger is looking for has been found. Registration in the lost item DB is performed when the robot 30 picks up the lost item or when the lost item is transported to the storage location. The above-mentioned candidate measures are merely examples, and some measures for lost items may be omitted, or measures not mentioned above may be taken.

[0054] In the handling information database, as handling candidates for the case where the category of the object is "a fallen object on the tracks", picking up the fallen object by the robot 30, discarding or storing the fallen object by the robot 30, and notifying station staff are described. Picking up refers to the robot 30 picking up and holding the fallen object on the tracks using the work tool 33. Discarding or storing refers to discarding the fallen object on the tracks if it is unnecessary, and storing it if it is necessary. The station staff is notified when the fallen object on the tracks is identified, when it is determined that the fallen object on the tracks cannot be picked up by the robot 30, or when the fallen object is picked up. The handling candidates described above are examples, and some of the handling measures for fallen objects on the tracks may be omitted, or handling measures not described above may be taken.

[0055] In the countermeasure information database, as countermeasure candidates for the case where the category of the object is "pilfered goods", the following are described: preventing the pilfering by the robot 30, stopping the pilfering by the robot 30, and notifying station staff. The prevention of the pilfering is performed for objects that may be pilfered. For example, if a bag is left behind, the robot 30 waits near the bag until the owner of the bag returns, thereby preventing the pilfering. The stopping of the pilfering is performed for objects that have been pilfered. For example, if a bag is pilfered, the robot 30 tracks the pilfered bag, grabs the bag with the work tool 33, or stops the pilfering using the speaker 36. The station staff is notified, for example, when the pilfering is detected. The above-mentioned countermeasure candidates are only examples, and some countermeasures for pilfered goods may be omitted, or countermeasures not described above may be performed.

[0056] In the countermeasure information database, as countermeasure candidates when the category of the object is "suspicious object", the following are described: calling attention to the suspicious object by the robot 30, isolating the suspicious object by the robot 30, and notifying station staff. Calling attention to the suspicious object is when the robot 30 moves to the position of the suspicious object and displays the presence of the suspicious object on the display unit 35, or uses the speaker 36 to notify by voice that there is a suspicious object. Isolating the suspicious object is when the suspicious object can be moved, picking it up and transporting it to a place where there are no passengers. The station staff is notified, for example, when the suspicious object is detected or when isolation of the suspicious object is completed. The above-mentioned countermeasure candidates are only examples, and some countermeasures for the suspicious object may be omitted, or countermeasures not described above may be performed for the suspicious object.

[0057] In the handling information database, cleaning by the robot 30 is described as a handling candidate when the category of the object is "dirt, dirt". The robot 30 uses the work tool 33 to clean the area where there is dirt or dirt, and removes the dirt or dirt. The handling candidate described above is an example, and other handling may be performed for the dirt or dirt.

[0058] In the countermeasure information database, as countermeasure candidates when the category of the object is "damaged equipment", repair by the robot 30, warning by the robot 30, and notification to station staff are described. Repair is when the robot 30 repairs the damaged equipment using the work tool 33. Warning is when the robot 30 located at the damaged equipment displays on the display unit 35 that the equipment is damaged, or notifies by voice using the speaker 36 that the equipment is damaged. Notification to station staff is performed, for example, when the damaged equipment is detected or when the repair of the damaged equipment is completed. The above-mentioned countermeasure candidates are only examples, and some countermeasures for the damaged equipment may be omitted, or countermeasures not described above may be performed for the damaged equipment.

[0059] Moreover, the process of step S106, particularly the process of selecting the necessary measures, may be performed using AI. As described above, the category and position of the object, and the analysis result of the sensor information are clearly related to the measures to be taken by the robot 30. Therefore, the measures to be taken by the robot 30 can be estimated using machine learning. For example, in the learning phase, a judgment model is constructed by associating the category and position of the object, the analysis result of the sensor information, and the measures taken or to be taken by the robot 30 using virtual cases or past cases in the station as learning data and performing machine learning. In the judgment phase, the category and position of the object, and the analysis result of the sensor information are input to the judgment model. The judgment model outputs the measures to be taken by the robot 30 based on the input data. Here, the judgment model learns the association between the various information and the measures to be taken by the robot 30 from various perspectives, and can accurately predict the measures to be taken by the robot 30.

[0060] By carrying out the above processing, the system can identify objects that require attention and have the robot 30 deal with the objects. This can significantly reduce the burden on station staff compared to when station staff deal with the objects themselves. In addition, since the flowchart in FIG. 2 is performed on images of various locations within the station premises, the system can perform maintenance on a wide area within the station premises. This can also reduce the burden on station staff of having to continually watch footage from multiple surveillance cameras.

[0061] (Feature 1) The station premises maintenance system 1 of this embodiment includes an image recognition unit 22a, a command generation unit 22c, and a robot 30. The image recognition unit 22a acquires an image generated by capturing an image of the station premises with the imaging device 11, and identifies the category and position of an object captured in the image by image recognition. The command generation unit 22c generates a command to deal with the object based on the category and position of the object identified by the image recognition unit 22a. The robot 30 receives the command generated by the command generation unit 22c, and moves to the position of the object based on the command, and deals with the object.

[0062] This makes it possible to deal with situations where there is an object inside the station. In particular, the decision on the necessary response and its execution are automated, significantly reducing the burden on station staff.

[0063] (Feature 2) The station premises maintenance system 1 of this embodiment includes an analysis unit 22b that acquires and analyzes at least one of heat information, sound information, three-dimensional detection information, and vibration information related to an object, which are generated by detecting the station premises with the sensor 12. The command generation unit 22c generates a command to deal with the object based on the category and position of the object specified by the image recognition unit 22a and the analysis result of the analysis unit 22b, and transmits the command to the robot 30.

[0064] This allows the robot 30 to respond more appropriately since it generates commands based on more detailed information about the object.

[0065] (Feature 3) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a acquires new images as time passes and identifies a new position of the identified object. The command generation unit 22c generates a command to move the robot 30 to the new position identified by the image recognition unit 22a and transmits the command to the robot 30.

[0066] As a result, even if the object moves, the robot 30 can be moved to the position of the object after the movement, so that the robot 30 can deal with the object even if it moves.

[0067] (Feature 4) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a identifies the position of an object by using a plurality of images generated by the plurality of imaging devices 11 capturing images of the station premises, respectively.

[0068] This allows the position of the object to be identified with high accuracy.

[0069] (Feature 5) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a performs image recognition on the image to determine that an object shown in the image is a lost item and to identify the location of the lost item. The command generation unit 22c generates a command to move the robot 30 to the location of the lost item and a command to have the robot 30 pick up the lost item, and transmits these to the robot 30.

[0070] This allows the system to recognize that there is a lost item within the station and use the robot 30 to pick it up.

[0071] (Feature 6) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a performs image recognition on the image to determine that an object shown in the image is dirt or dirt and to identify the position of the dirt or dirt. The command generation unit 22c generates and transmits to the robot 30 a command to move the robot 30 to the position of the dirt or dirt and a command to the robot 30 to remove the dirt or dirt.

[0072] This allows the system to recognize that there is trash or dirt inside the station and to use the robot 30 to clean it up.

[0073] (Feature 7) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a identifies an object shown in an image as a suspicious object and identifies the location of the suspicious object by image recognition of the image. The command generation unit 22c generates and transmits to the robot 30 a command to move the robot 30 to the location of the suspicious object and a command to alert the robot 30 to the suspicious object or to isolate the suspicious object.

[0074] This allows the system to recognize the presence of a suspicious object within the station premises and to deal with the suspicious object using the robot 30.

[0075] (Feature 8) In the station premises maintenance system 1 of this embodiment, the image recognition unit 22a notifies a station staff member when it is determined that the category or position of an object shown in an image satisfies a notification condition.

[0076] This makes it possible to notify station staff, for example, when it is difficult for the robot 30 to handle the situation alone, or when the category or location of the object is highly important.

[0077] (Feature 9) In the station premises maintenance system 1 of this embodiment, the storage that stores images generated by capturing images of the station premises with the imaging device 11, the image recognition unit 22a, and the command generation unit 22c are all provided on the cloud.

[0078] This allows images captured inside multiple stations to be handled by a single system.

[0079] The above-mentioned features 1 to 9 can be combined, for example, as follows to realize the station premises maintenance system 1. The same applies to the station premises maintenance method or the station premises maintenance system program. [Configuration 1] Station premises maintenance system 1 with feature 1 [Configuration 2] Station premises maintenance system 1 with feature 2 in addition to configuration 1 [Configuration 3] Station premises maintenance system 1 having feature 3 in addition to configuration 1 or 2 [Configuration 4] Station premises maintenance system 1 having feature 4 in addition to any one of configurations 1 to 3 [Configuration 5] Station premises maintenance system 1 having feature 5 in addition to any one of configurations 1 to 4 [Configuration 6] In addition to any one of configurations 1 to 5, a station premises maintenance system 1 further having feature 6 [Configuration 7] Station premises maintenance system 1 having any one of configurations 1 to 6, and further having feature 7 [Configuration 8] In addition to any one of configurations 1 to 7, a station premises maintenance system 1 further having feature 8 [Configuration 9] In addition to any one of configurations 1 to 8, a station premises maintenance system 1 further having feature 9

[0080] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general purpose processors, special purpose processors, integrated circuits, Application Specific Integrated Circuits (ASICs), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. Processors are considered processing circuits or circuits because they include transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or the processor. [Explanation of symbols]

[0081] 1 Station premises maintenance system 22 Processing Server 22a Image recognition section 22b Analysis Department 22c Command generator 30. Robot

Claims

1. an image recognition unit that acquires an image generated by capturing an image of the inside of a station using an imaging device and identifies a category and a position of an object shown in the image by image recognition; a command generating unit that generates a command for dealing with the object based on the category and position of the object identified by the image recognition unit; a robot that receives the command generated by the command generation unit, moves to a position of the object based on the command, and deals with the object; This is an in-station maintenance system.

2. The station premises maintenance system according to claim 1, An analysis unit that acquires and analyzes at least one of heat information, sound information, three-dimensional detection information, and vibration information related to the object, the information being generated by detecting the inside of the station with a sensor; A station premises maintenance system, wherein the command generation unit generates a command for dealing with the object based on the category and position of the object identified by the image recognition unit and the analysis results of the analysis unit, and transmits the command to the robot.

3. The station premises maintenance system according to claim 1, The image recognition unit acquires new images over time and identifies a new position of the identified object; The command generation unit generates a command to move the robot to a new position identified by the image recognition unit and transmits the command to the robot.

4. The station premises maintenance system according to claim 1, The image recognition unit identifies the position of the object using a plurality of images generated by capturing images of the station premises with a plurality of the imaging devices.

5. The station premises maintenance system according to claim 1, the image recognition unit determines that the object shown in the image is a lost item and identifies a location of the lost item by performing image recognition on the image; The command generating unit generates and transmits to the robot a command to move the robot to the location of the lost item and a command to have the robot pick up the lost item.

6. The station premises maintenance system according to claim 1, the image recognition unit identifies the object shown in the image as dust or dirt and a position of the dust or dirt by performing image recognition on the image; The command generating unit generates and transmits to the robot a command to move the robot to the location of the garbage or dirt, and a command to the robot to remove the garbage or dirt.

7. The station premises maintenance system according to claim 1, The image recognition unit identifies an object shown in the image as a suspicious object and a position of the suspicious object by performing image recognition on the image, The command generation unit generates and transmits to the robot a command to move the robot to the location of the suspicious object and a command to have the robot alert the robot to the suspicious object or isolate the suspicious object.

8. The station premises maintenance system according to claim 1, The station premises maintenance system, wherein the image recognition unit notifies station staff when it determines that the category or position of the object shown in the image satisfies a notification condition.

9. The station premises maintenance system according to claim 1, A station premises maintenance system, in which a storage for storing images generated by an imaging device capturing images of a station premises, the image recognition unit, and the command generation unit are all provided on the cloud.

10. An image is acquired by capturing an image of the inside of a station using an imaging device, and a category and a position of an object shown in the image are identified by image recognition; generating instructions to a robot to deal with the object based on the category and position of the object identified by the image recognition, and transmitting the instructions to the robot; The station premises maintenance method includes moving the robot that has received the command to the location of the object and having the robot deal with the object.

11. An image is acquired by capturing an image of the inside of a station using an imaging device, and a category and a position of an object shown in the image are identified by image recognition; generating a command to deal with the object based on the category and position of the object identified by the image recognition, and transmitting the command to the robot; A station maintenance program that causes a computer to carry out processing.

Images