Periphery monitoring method, periphery monitoring system, and vehicle

Abstract

According to the present invention, periphery information that is detected by a sensor (20) provided in the vehicle (1) and includes situations in a periphery of a vehicle (1) is acquired, a parking position where the vehicle (1) is parked is stored in advance, determination regarding whether or not there is a visitor visiting a building located in the periphery of the parking position, is made on the basis of periphery information including situations in the periphery of the vehicle (1) parked at the parking position, and when determination is made that a visitor is present, notification information regarding the visitor is notified to a user of the vehicle or a person associated with the building.

Description

Surroundings monitoring method, surroundings monitoring system, and vehicle

[0001] The present invention relates to a surroundings monitoring method, a surroundings monitoring system, and a vehicle.

[0002] A technology is known that, based on information about the vehicle's condition, determines situations in which actions by a third party could pose a danger to the vehicle or personnel associated with the vehicle, selects an appropriate notification method, and proposes measures to avoid the danger (Patent Document 1).

[0003] JP 2005-227965 A

[0004] However, the technology described in Patent Document 1 is an invention that determines the presence of a suspicious person around the vehicle and notifies the driver when such a person is present, and has the problem that even if there is a visitor visiting a building located around the parking position of the vehicle, information about the visitor cannot be notified to the user of the vehicle or a person associated with the building.

[0005] The problem to be solved by the present invention is to provide a surroundings monitoring method, a surroundings monitoring system, and a vehicle that can notify the vehicle user or a person associated with the building of information regarding visitors to a building located around the parking position of the vehicle.

[0006] The present invention solves the above problem by acquiring surrounding information including the situation around the vehicle detected by a sensor installed in the vehicle, pre-storing the parking location where the vehicle will be parked, determining whether there are visitors visiting a building located around the parking location based on the surrounding information including the situation around the vehicle parked in the parking location, and if it is determined that there are visitors, notifying the user of the vehicle or a person associated with the building of notification information regarding the visitor.

[0007] According to the present invention, information regarding visitors to a building located around the parking position of a vehicle can be notified to the user of the vehicle or a person associated with the building.

[0008] Fig. 1 is a block diagram showing an example of the configuration of a surroundings monitoring system including a vehicle according to an embodiment of the present invention. Fig. 2 is a diagram showing an example of a screen for registering a predetermined moving direction according to the embodiment. Fig. 3 is a flowchart showing the processing flow of a surroundings monitoring method executed by a controller according to the embodiment.

[0009] A surroundings monitoring system including a vehicle according to an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating an example of the configuration of the surroundings monitoring system including a vehicle according to an embodiment of the present invention. As illustrated in FIG. 1 , the surroundings monitoring system 100 includes a vehicle 1, a terminal device 2, and a network 3. The vehicle 1 and the terminal device 2 exchange information with each other via wireless communication via the network 3. The terminal device 2 is a device operated by a user outside the vehicle. The terminal device 2 is, for example, a terminal device installed in a building or a portable terminal device such as a smartphone. The user is, for example, the owner of the vehicle 1. In this embodiment, when the vehicle 1 is parked in a predetermined parking position, the vehicle 1 determines whether there is a visitor visiting a building located around the parking position based on surrounding information including the situation around the vehicle 1 detected by a sensor 20 provided in the vehicle 1, and notifies the user of the vehicle of notification information regarding the visitor. Note that in this embodiment, the notification information is not limited to the user of the vehicle, and may also be notified to a person associated with the building.

[0010] For example, the building is the user's home (the user's home), and the parking location is the user's parking lot. The user returns home in vehicle 1, parks vehicle 1 in the parking lot, and enters the home. The parked vehicle 1 determines whether there is a visitor at the home based on surrounding information, and if there is a visitor, notifies the user who is inside the home of notification information. For example, the notification information is notified to the user via a terminal device at the home or a terminal device owned by the user. Note that the building is not limited to the user's home, but may be any building located around the parking location, such as a building where the user's workplace is located. Furthermore, the parking location is not limited to the parking lot at the user's home, but may also be a parking lot contracted by the user's workplace. Furthermore, not only when the user is at home, but also when the user is located away from home, such as at the user's workplace, when the vehicle 1 is parked in the parking lot at the home, the vehicle 1 may monitor visitors to the home and notify the user of the vehicle or a person associated with the building of the notification information. The person associated with the building may be the owner or manager of the building, an employee of the company to which the building belongs, or a temporary user of the building. For example, the building may be a variety of buildings, such as a corporate office, a sports facility, a workspace, or a public facility such as a library.

[0011] The vehicle 1 includes a controller 10, a sensor 20, an output device 30, an input device 40, and a communication device 50. These devices are connected to each other via a Controller Area Network (CAN) or other in-vehicle LAN for transmitting and receiving information to and from each other.

[0012] The sensor 20 is a device provided in the vehicle 1 and detects the situation around the vehicle 1. The sensor 20 detects the surrounding environment, including objects present around the vehicle 1. The objects include humans such as pedestrians. The objects may be any moving, dynamic objects, and may also be animals other than humans. Surrounding information, including the situation around the vehicle 1, detected by the sensor 20 is output to the controller 10. In this embodiment, the sensor 20 is a camera equipped with an imaging element such as a CCD and is mounted on the vehicle 1. The sensor 20 may include a front camera, a rear camera, and / or a side camera of the vehicle 1. The sensor 20 captures images of the situation around the vehicle 1. The sensor 20 may also include a radar device such as a millimeter-wave radar, a laser radar, or an ultrasonic radar. The radar device detects surrounding objects, such as pedestrians, based on received signals.

[0013] The output device 30 is an interface for outputting information. Examples of the output device 30 include, but are not limited to, a display that displays text information or image information and a speaker that outputs sound such as music or voice. The output device 30 is a display configured with a liquid crystal panel, an organic EL panel, or the like. The output device 30 outputs image information and audio information acquired from the terminal device 2. The image information is, for example, image information of a user captured by the terminal device 2. The audio information is, for example, audio information recorded by the terminal device 2 of the user's voice. The output device 30 is provided on the outer side of the vehicle 1. The output device 30 is used by visitors to communicate with remote users via the terminal device 2.

[0014] The input device 40 accepts various information input by the user. The input device 40 is, for example, a switch or a touch panel. If the input device 40 is a touch panel, it may also serve as the output device 30. The input device 40 outputs the input information to the controller 10. In this embodiment, the information input to the input device 40 is registration information registered in advance by the user. The registration information is registered in advance before the determination process described below is executed. The registration information includes information about a parking location for parking the vehicle 1. For example, the user registers a parking space at home as the parking location. Specifically, when the user parks the vehicle 1 in a parking location that the user wants to register, the user registers the parking location on a registration screen displayed on the output device 30. The registration screen displays the current location of the vehicle 1, i.e., the parking location of the vehicle 1, on a map. The user confirms the displayed current location of the vehicle 1 and registers the current location of the vehicle 1 as the parking location. Note that the method of registering the parking location is not limited to this. For example, the method may involve inputting coordinate values ​​(latitude and longitude) of a location to be registered as a parking location from map information. The registered information may also include surrounding information about the parking position. The surrounding information includes surrounding images captured around the parking position. The surrounding information may also include information about the locations of buildings around the parking position. These buildings are the targets for determining whether or not there are visitors. By registering information about the locations of the buildings to be determined, the user can specify which building to determine whether or not there are visitors when there are multiple buildings located around the parking position.

[0015] The registration information includes information on a predetermined movement direction used to determine visitors. In this embodiment, information on a predetermined movement direction associated with each parking location is registered for each parking location. The predetermined movement direction is a direction toward a building located around the parking location. The predetermined movement direction is a direction along a route to the building, for example, a direction from the entrance to the site where the building is located to the building's front door. For example, when a surrounding image including a building is displayed on the output device 30, the user inputs and registers the predetermined movement direction by tracing a line on the surrounding image with their finger.

[0016] Here, the predetermined movement direction to be registered will be described. FIG. 2 is a diagram showing an example of a screen for registering the predetermined movement direction in this embodiment. The description will be given assuming that the parking location is a parking lot P and the building is a home B. In FIG. 2, as an example of a screen displayed on the output device 30, a surrounding image SI including the surrounding conditions of the parking lot P where the vehicle V1 is parked is shown. In the example of FIG. 2, the parking lot P is located within the site S of the home B. The site S faces a road R. The user checks the surrounding image SI displayed on the output device 30 and registers the predetermined movement direction. For example, the user touches any position on the surrounding image SI with a finger to set the starting point of the predetermined movement direction. Then, the user traces their finger in a line from the starting point, thereby registering the trajectory traced by the finger as the predetermined movement direction. In the example of FIG. 2, the input predetermined movement direction is represented by an arrow D. Alternatively, the user may touch the starting point and end point of the predetermined movement direction (the starting point and end point of the arrow in FIG. 2) with a finger to register the direction from the starting point to the end point as the predetermined movement direction D.

[0017] The predetermined movement direction may also be registered as a direction based on the position and / or orientation of the vehicle 1. In this embodiment, the sensor 20 provided in the vehicle 1 is used to determine whether a pedestrian around the vehicle 1 is a visitor, and therefore the predetermined movement direction is registered based on the position of the vehicle 1. For example, in the example of FIG. 2 , when the vehicle 1 is parked backward, the predetermined movement direction is registered as a direction from the front to the rear of the vehicle 1 on the left side of the vehicle 1. In this case, during the determination process, visitors are monitored by a side sensor located on the left side of the vehicle 1. Note that when the vehicle 1 is parked forward, the predetermined movement direction is registered as a direction opposite to that in the case of backward parking. Specifically, the predetermined movement direction shown in FIG. 2 is registered as a direction from the rear to the front of the vehicle 1 on the right side of the vehicle 1. The predetermined movement direction is not limited to one direction, and may be set as a combination of two or more different directions. For example, the predetermined movement direction may be set as a predetermined route. The user can set a location and a direction to move according to the route.

[0018] The registration information may also include information about a predetermined person. The predetermined person is a person registered in advance by the user as a potential visitor. The predetermined person may be, for example, the user's friend, family member, or other acquaintance, or a prospective visitor. The user pre-registers information including the characteristics of the predetermined person in association with identification information that identifies the predetermined person, such as a name. The characteristics of the predetermined person may be, for example, characteristics related to appearance, such as face or clothing.

[0019] The communication device 50 is connected to the terminal device 2 via the network 3 and transmits and receives information. Specifically, the communication device 50 transmits image information and / or audio information to the terminal device 2 and receives image information and / or audio information from the terminal device 2. For example, in this embodiment, the communication device 50 outputs notification information to the terminal device 2.

[0020] The controller 10 monitors the surroundings of the vehicle 1 based on surrounding information including the conditions around the vehicle 1 detected by the sensor 20. The controller 10 is composed of a ROM (Read Only Memory) that stores a program for monitoring the surroundings of the vehicle 1, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The program may be stored in a location outside the controller 10, such as a computer-readable recording medium.

[0021] The controller 10 includes, as functional blocks, at least an information acquisition unit 101 that acquires various types of information, a storage unit 102 that stores registration information, a determination unit 103 that determines whether or not there is a visitor based on the acquired information, and a notification unit 104 that notifies the user of the determination result. These blocks realize the functions described below using programs stored in ROM. Each functional block executes its function through cooperation between software and hardware (processor). Note that the functional blocks of the controller 10 do not necessarily need to be divided into functional blocks as shown in FIG. 1 , and the number of functional blocks is not limited.

[0022] The information acquisition unit 101 executes an information acquisition process to acquire various types of information. In the information acquisition process, the information acquisition unit 101 acquires surrounding information including the situation around the vehicle 1 from the sensor 20. The surrounding information is, for example, a surrounding image capturing the situation around the vehicle 1. The surrounding image includes a front surrounding image capturing an image of the front of the vehicle 1, a side surrounding image capturing an image of the side of the vehicle 1, and a rear surrounding image capturing an image of the rear of the vehicle 1. The information acquisition unit 101 may also generate an AVM image for an around view monitor as the surrounding image by combining the surrounding images captured by multiple cameras. In this embodiment, when the vehicle 1 is parked in a parking position stored in advance, the information acquisition unit 101 acquires surrounding information including the situation around the vehicle 1 parked in the parking position. For example, the information acquisition unit 101 acquires the current position of the vehicle 1. If the current position of the vehicle 1 matches the parking position, the information acquisition unit 101 determines that the vehicle 1 is parked in the parking position and acquires surrounding information from the sensor 20.

[0023] Furthermore, when acquiring surrounding information, the information acquisition unit 101 may acquire the surrounding information according to a predetermined movement direction that is registered in advance. For example, if the predetermined movement direction is registered based on the position of the vehicle 1, the information acquisition unit 101 acquires the surrounding information according to the position where the predetermined movement direction is set. Specifically, as shown in FIG. 2 , if the predetermined movement direction is set to the front and left lateral areas of the vehicle 1, a front surrounding image capturing an area in front of the vehicle 1 and a left surrounding image capturing an area in the left lateral area of ​​the vehicle 1 are acquired. Furthermore, the information acquisition unit 101 may first activate a camera in the direction toward the road among the front, side, and rear of the vehicle 1 to acquire surrounding images, and if a pedestrian is recognized from the surrounding image, activate the remaining camera to acquire surrounding images to determine the pedestrian's movement direction. Specifically, as shown in FIG. 2 , if the vehicle V1 is parked backward and a road R is in front of the vehicle 1, the front camera of the vehicle 1 is activated first to acquire surrounding images in front of the vehicle 1. Alternatively, the information acquisition unit 101 may first activate a radar device to acquire surrounding information, and when a pedestrian is detected by the radar device, activate a camera to acquire a surrounding image.

[0024] Furthermore, the information acquisition unit 101 acquires registration information from the input device 40. The registration information includes, for example, information on the parking position and information on a predetermined movement direction for determining a visitor. When the vehicle 1 is parked in a parking position, the information acquisition unit 101 acquires information on the predetermined movement direction associated with the parking position.

[0025] Furthermore, in this embodiment, when the determination unit 103 determines that the visitor belongs to a specific carrier, the information acquisition unit 101 may acquire information about the package that the visitor brought to the building from a shipping information database managed by the specific carrier. A carrier includes, for example, a corporation that operates a shipping business such as postal or courier services. A person who belongs to a specific carrier includes, for example, a postal worker, a delivery person, or another person employed by the carrier. The shipping information database stores information about each package, such as the sender, destination, and type of package. Package types include types such as postal mail and courier services. The information acquisition unit 101 identifies a package whose destination is specified by the user from the shipping information database and acquires information about the package associated with the package.

[0026] The information acquisition unit 101 may also acquire user information about the user. The user information may include, for example, information indicating whether the user is a subscriber to a predetermined paid service. The paid service is a paid service that performs the perimeter monitoring process of the present application. The paid service monitors visitors to buildings located around each of two or more parking locations. The user may also subscribe to a free service to receive the service that performs the perimeter monitoring process of the present application. The free service allows the user to register one parking location and monitors visitors to buildings located around the registered parking location.

[0027] Furthermore, the information acquisition unit 101 may acquire vehicle information including the state of the vehicle 1. The vehicle information includes information on the shift position of the vehicle 1 and information on whether the start switch of the vehicle 1 is on or off. The shift position is, for example, the parking position or the drive position. For example, when it is determined that the vehicle 1 is parked in a parking position, the information acquisition unit 101 acquires the vehicle information at regular intervals.

[0028] The storage unit 102 executes a storage process for storing the registration information acquired from the input device 40. In the storage process, the storage unit 102 stores in advance information about the parking position where the vehicle 1 is to be parked. The storage unit 102 may store in advance a predetermined person.

[0029] The determination unit 103 executes a determination process. In the determination process, the determination unit 103 determines whether or not there is a visitor visiting a building located around the vehicle 1, based on surrounding information including the situation around the vehicle 1 parked in the parking position.

[0030] For example, the determination unit 103 estimates the movement direction of pedestrians around the building based on the acquired surrounding information. Then, if there is a pedestrian whose estimated movement direction matches a predetermined movement direction, the determination unit 103 determines that there is a visitor. If there is no pedestrian whose estimated movement direction matches the predetermined movement direction, the determination unit 103 determines that there is no visitor.

[0031] Here, as a specific example of the determination process, a determination process using a surrounding image will be described. When the vehicle 1 is parked in a parking position, the determination unit 103 performs image recognition processing on a surrounding image capturing the situation around the vehicle 1, and recognizes pedestrians present around the vehicle 1. The determination unit 103 estimates the movement direction in which the recognized pedestrian is moving. The determination unit 103 determines whether the estimated movement direction of the pedestrian matches a predetermined movement direction registered in advance. If the movement direction of the pedestrian matches the predetermined movement direction, the determination unit 103 determines that there is a visitor. If there is no pedestrian around the vehicle 1, or if the movement direction of the pedestrian does not match the predetermined movement direction, the determination unit 103 determines that there is no visitor.

[0032] Furthermore, if there is a visitor, the determination unit 103 may determine the type of visitor based on the surrounding information. The type of visitor includes, for example, child, adult, gender, employee of a company related to the building, and animal. For example, the determination unit 103 performs image recognition processing on the surrounding image to extract characteristics of the visitor and determine the type of visitor based on the extracted characteristics of the visitor. It may also determine whether the visitor is a suspicious person based on the visitor's clothing, behavior, and belongings. The determination unit 103 may also determine whether the visitor is a specified person. For example, the determination unit 103 performs image recognition processing on the surrounding image to extract characteristics of the visitor. If the extracted characteristics of the visitor match the characteristics of a pre-registered specified person, the determination unit 103 determines that the visitor is a specified person. If the extracted characteristics of the visitor do not match the characteristics of a pre-registered specified person, the determination unit 103 determines that the visitor is not a specified person. In this case, the visitor may be determined to be a suspicious person.

[0033] The determination unit 103 may execute a determination process of whether or not a visitor is present depending on the state of the vehicle 1. The determination unit 103 determines whether or not a visitor is present when the state of the vehicle 1 is such that the shift position of the vehicle 1 is in the parking position or the start switch of the vehicle 1 is in the off state. For example, the determination unit 103 starts the determination process of whether or not a visitor is present when the shift position of the vehicle 1 is in the parking position or the start switch of the vehicle 1 is in the off state after the vehicle 1 is parked in the parking position. Then, while the shift position of the vehicle 1 is in the parking position or the start switch of the vehicle 1 is in the off state, the determination unit 103 repeatedly executes the determination process at regular intervals. Furthermore, the determination unit 103 stops determining whether or not a visitor is present when the state of the vehicle 1 is such that the shift position of the vehicle 1 is in the drive position or the start switch of the vehicle 1 is in the on state.

[0034] When there is a visitor, the determination unit 103 may determine whether the visitor belongs to a specific delivery company based on the surrounding information. For example, the determination unit 103 performs image recognition processing on the surrounding image to extract characteristics of the visitor. If the characteristics of the visitor match the characteristics of a person who belongs to a specific delivery company, the determination unit 103 determines that the visitor belongs to the specific delivery company. A person who belongs to a specific delivery company is, for example, a postal worker or a delivery person. The determination unit 103 identifies the visitor's clothing from the visitor's characteristics and determines that the visitor is a postal worker or a delivery person. The clothing of postal workers and delivery people varies depending on the delivery company. Clothing characteristics for each delivery company are stored in advance. The determination unit 103 identifies which delivery company the visitor belongs to based on the visitor's clothing.

[0035] The determination unit 103 may determine whether the user is a subscriber who subscribes to a predetermined paid service based on the user information of the user. The user information includes information indicating whether the user is a subscriber. If the determination unit 103 determines that the user is a subscriber, it sets the system to a state where two or more parking locations can be registered. Furthermore, if the determination unit 103 determines that the user is not a subscriber, it sets the system to a state where one parking location can be registered.

[0036] When the determination unit 103 determines that there is a visitor, the notification unit 104 notifies the user of notification information regarding the visitor. The notification information may also include the type of visitor and / or information indicating that the visitor is a specific person. The notification information may also include route and package information. The package information may include the contents of the package, the arrival date and time, and the sender of the package. The notification information is represented by an image or text. In this embodiment, the notification unit 104 transmits the notification information to the terminal device 2 via the communication device 50. When the terminal device 2 receives the notification information, it outputs the notification information. The notification unit 104 may also transmit the notification information to a server managed by a security company or the like.

[0037] When it is determined that there is a visitor, the notification unit 104 may connect the output device 30 provided in the vehicle 1 and the terminal device 2 used by the user in a state where communication is possible between them. In the communication-possible state, the notification unit 104 causes the output device 30 to output image information and / or audio information of the user, and causes the terminal device 2 to output image information and / or audio information of the visitor. As described above, the notification unit 104 causes the output device 30 and the terminal device 2 to transmit and receive image information and audio information of the communication destination to each other, thereby executing a videophone call function.

[0038] The terminal device 2 is a device that outputs information to a user. When notification information is received from the vehicle 1, the terminal device 2 outputs the notification information. The terminal device 2 has an interface for outputting information. For example, the terminal device 2 has a display that displays image information and a speaker that outputs sound such as music or voice, but is not limited to these. The display is configured, for example, with a liquid crystal panel, an organic EL panel, or the like. The terminal device 2 outputs the notification information acquired from the vehicle 1 as image information and / or audio information. The terminal device 2 may also be used for communication between a user and a visitor with the vehicle 1. When the terminal device 2 receives image information and / or audio information from the vehicle 1, it outputs the image information and / or audio information. The image information received from the vehicle 1 is image information of a visitor captured by the sensor 20 of the vehicle 1. The audio information received from the vehicle 1 is audio information of the visitor's voice recorded by the sensor 20 of the vehicle 1. The terminal device 2 also transmits the image information and / or audio information to the vehicle 1. The image information is, for example, image information of the user captured by the terminal device 2. The audio information is, for example, audio information of the user's voice recorded by the terminal device 2.

[0039] The terminal device 2 may accept input of various information by the user. The terminal device 2 includes, for example, a switch or a touch panel. In this embodiment, the terminal device 2 may accept input of registration information by the user together with or instead of the input device 40 of the vehicle 1. The terminal device 2 transmits the input registration information to the vehicle 1.

[0040] The network 3 is a network outside the vehicle 1 and refers to a telecommunications network such as the Internet or a LAN. There are no particular limitations on the network 3, and any known network can be used as long as it allows information to be exchanged between the vehicle 1 and the terminal device 2. The network 3 includes a network within a building or on the premises of a building. The network within the building or on the premises is connected to the terminal device 2 within the building or on the premises. The network within the building or on the premises of a building may use Wi-Fi (registered trademark). In this embodiment, the communication device 50 of the vehicle 1 transmits notification information to the terminal device 2 within the building or on the premises via the network within the building or on the premises of the building.

[0041] Next, an example of a surroundings monitoring method executed by the controller according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the processing flow of the surroundings monitoring method executed by the controller according to this embodiment.

[0042] In step S1, the controller 10 pre-stores the parking position where the vehicle 1 will be parked. For example, if the user registers a parking position via the input device 40, the controller 10 stores the registered parking position. In step S2, the controller 10 determines whether the vehicle 1 has been parked in the parking position. The controller 10 compares information about the registered parking position with information about the current position of the vehicle 1, and if the current position of the vehicle 1 matches the parking position, the controller 10 determines that the vehicle 1 has been parked in the parking position. If it is determined that the vehicle 1 has been parked in the parking position, the controller 10 proceeds to step S3. If it is determined that the vehicle 1 has not been parked in the parking position, the controller 10 returns to step S2 and repeats the subsequent flow.

[0043] In step S3, the controller 10 acquires surrounding information. The acquired surrounding information includes the situation around the vehicle 1 parked in the parking position. The surrounding information is, for example, a surrounding image capturing the situation around the vehicle 1. In step S4, the controller 10 determines whether or not there is a visitor based on the surrounding information. For example, if the movement direction of a pedestrian around the vehicle 1 matches a predetermined movement direction based on the surrounding information, the controller 10 determines that there is a visitor, assuming that the pedestrian is a visitor. If it is determined that there is a visitor, the controller 10 proceeds to step S5. If it is determined that there is no visitor, the controller 10 returns to step S3 and repeats the subsequent steps. In step S5, the controller 10 notifies the user of notification information. For example, the controller 10 transmits the notification information to the terminal device 2 operated by the user. When the terminal device 2 receives the notification information, it outputs the notification information as image information and / or audio information.

[0044] In the flowchart of FIG. 3 , the processing of steps S3 to S5 (surroundings monitoring processing) is executed when it is determined in step S2 that the vehicle 1 has been parked in a parking position. However, this embodiment is not limited to this. For example, the surroundings monitoring processing may be started when the ignition switch of the vehicle 1 is turned off after it is determined that the vehicle 1 has been parked in a parking position. While the surroundings monitoring processing is being executed, the surroundings monitoring mode is set. Furthermore, after the surroundings monitoring processing is started, the controller 10 periodically repeatedly executes a process for determining whether or not a visitor is present. Then, when the ignition switch of the vehicle 1 is turned on, the controller 10 may end the surroundings monitoring mode and stop the surroundings monitoring processing.

[0045] As described above, in the surroundings monitoring method, surroundings monitoring system, and vehicle according to this embodiment, the controller acquires surrounding information including the situation around the vehicle detected by a sensor provided in the vehicle, stores in advance the parking position where the vehicle will be parked, determines whether there is a visitor visiting a building located around the parking position based on the surrounding information including the situation around the vehicle parked in the parking position, and if it determines that there is a visitor, notifies the user of the vehicle or a person associated with the building of notification information about the visitor. In this way, information about visitors to buildings located around the parking position of the vehicle can be notified to the user of the vehicle or a person associated with the building.

[0046] Furthermore, in the surroundings monitoring method, surroundings monitoring system, and vehicle according to the present embodiment, when the controller determines that a visitor is present, it determines the type of visitor based on the surroundings information and notifies the user of the vehicle or a person associated with the building of notification information including the type of visitor, thereby enabling the user of the vehicle or a person associated with the building to be notified of the type of visitor.

[0047] Furthermore, in the surroundings monitoring method, surroundings monitoring system, and vehicle according to the present embodiment, the controller stores a predetermined person in advance, and when it determines that a visitor is present, it determines whether the visitor is the predetermined person, and when it determines that the visitor is the predetermined person, it notifies the vehicle user or a person associated with the building of notification information including that the visitor is the predetermined person. In this way, when the visitor is the predetermined person, it is possible to notify the vehicle user or a person associated with the building that the visitor is the predetermined person.

[0048] Furthermore, in the perimeter monitoring method, perimeter monitoring system, and vehicle according to this embodiment, when the controller determines that there is a visitor, it determines based on the perimeter information whether the visitor belongs to a specific carrier, and if it determines that the visitor belongs to a specific carrier, it obtains information about the package that the visitor brought to the building from a transportation information database managed by the specific carrier, and notifies the vehicle user or a person associated with the building of notification information including the package information. This allows the vehicle user or a person associated with the building to be notified of the package information.

[0049] Furthermore, in the surroundings monitoring method, surroundings monitoring system, and vehicle according to this embodiment, when the controller determines that a visitor is present, the controller connects the output device provided in the vehicle to a terminal device used by the vehicle user or a person associated with the building so that they can communicate with each other, causes the output device to output image information and audio information of the vehicle user or the person associated with the building, and causes the terminal device to output image information and audio information of the visitor, thereby enabling the vehicle user or the person associated with the building to communicate with the visitor outside the building.

[0050] In the surroundings monitoring method, surroundings monitoring system, and vehicle according to the present embodiment, the controller determines whether the user is a subscriber to a predetermined paid service based on the user information of the user, and if it determines that the user is a subscriber, sets the system to a state in which two or more parking positions can be registered, and if it determines that the user is not a subscriber, sets the system to a state in which one parking position can be registered, and stores the parking position registered by the user in advance. This makes it possible to change the number of parking positions that can be registered depending on whether the user is a subscriber to a paid service.

[0051] In the surroundings monitoring method, surroundings monitoring system, and vehicle according to this embodiment, the controller acquires vehicle information including the vehicle state, and determines whether or not a visitor is present when the vehicle state indicates that the shift position of the vehicle is in the park position or the vehicle start switch is off, and stops determining whether or not a visitor is present when the vehicle state indicates that the shift position of the vehicle is in the drive position. This makes it possible to determine whether or not a visitor is present depending on the vehicle state.

[0052] Furthermore, in the surroundings monitoring method, surroundings monitoring system, and vehicle according to this embodiment, the controller stores a predetermined movement direction for determining visitors for each building, estimates the movement direction of pedestrians around the building based on surrounding information including the situation around the vehicle parked in the parking space, and determines that there is a visitor if the estimated movement direction of a pedestrian matches the predetermined movement direction. This makes it possible to identify visitors to the building from among pedestrians around the vehicle.

[0053] It should be noted that the above-described embodiments have been described to facilitate understanding of the present invention, and are not intended to limit the present invention. Therefore, each element disclosed in the above-described embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0054] REFERENCE SIGNS LIST 1 vehicle 20 sensor 10 controller 101 information acquisition unit 102 storage unit 103 determination unit 104 notification unit 2 terminal device 3 network

Claims

1. A surrounding monitoring method executed by a controller, wherein the controller acquires surrounding information including the situation around the vehicle detected by a sensor provided in the vehicle, stores in advance a parking position where the vehicle parks, determines whether there is a visitor who visits a building located around the parking position based on the surrounding information including the situation around the vehicle parked at the parking position, and when it is determined that there is a visitor, notifies notification information regarding the visitor to a user of the vehicle or a person related to the building.

2. The surrounding monitoring method according to claim 1, wherein the controller determines the type of the visitor based on the surrounding information when it is determined that there is a visitor, and notifies the notification information including the type of the visitor to a user of the vehicle or a person related to the building.

3. The surrounding monitoring method according to claim 1 or 2, wherein the controller stores in advance a predetermined person, determines whether the visitor is the predetermined person when it is determined that there is a visitor, and when it is determined that the visitor is the predetermined person, notifies the notification information including that the visitor is the predetermined person to a user of the vehicle or a person related to the building.

4. The surrounding monitoring method according to any one of claims 1 to 3, wherein the controller determines whether the visitor belongs to a specific carrier based on the surrounding information when it is determined that there is a visitor, and when it is determined that the visitor belongs to the specific carrier, acquires information on the luggage carried by the visitor to the building from a transportation information database managed by the specific carrier, and notifies the notification information including the luggage information to a user of the vehicle or a person related to the building.

5. The surrounding monitoring method according to any one of claims 1 to 4, wherein when the controller determines that the visitor is present, it connects to a state where communication is possible between an output device provided in the vehicle and a terminal device used by the user of the vehicle or a person related to the building, and causes the output device to output image information and voice information of the user of the vehicle or the person related to the building, and causes the terminal device to output image information and voice information of the visitor.

6. The surrounding monitoring method according to any one of claims 1 to 5, wherein the controller determines whether the user is a subscriber who subscribes to a predetermined paid service based on the user information of the user, and when it is determined that the user is the subscriber, sets a state in which registration of two or more of the parking positions is possible, and when it is determined that the user is not the subscriber, sets a state in which registration of one parking position is possible, and stores in advance the parking position registered by the user.

7. The surrounding monitoring method according to any one of claims 1 to 6, wherein the controller acquires vehicle information including the state of the vehicle, and when the state of the vehicle is a state where the shift position of the vehicle is in the parking position or the start switch of the vehicle is off, determines whether the visitor is present, and when the state of the vehicle is a state where the shift position of the vehicle is in the drive position, stops determining whether the visitor is present.

8. The surrounding monitoring method according to any one of claims 1 to 7, wherein the controller stores a predetermined moving direction for determining the visitor for each building, estimates the moving direction of a pedestrian around the building based on the surrounding information including the situation around the vehicle parked at the parking position, and determines that the visitor is present when there is a pedestrian whose estimated moving direction matches the predetermined moving direction.

9. A surrounding monitoring system comprising a vehicle and a controller, wherein the controller acquires surrounding information including the situation around the vehicle detected by a sensor provided in the vehicle, stores in advance a parking position where the vehicle parks, determines whether there is a visitor visiting a building located around the parking position based on the surrounding information including the situation around the vehicle parked at the parking position, and when it is determined that there is a visitor, notifies notification information regarding the visitor to a user of the vehicle or a person related to the building.

10. A vehicle comprising a sensor and a controller, wherein the controller acquires surrounding information including the situation around the vehicle detected by the sensor, stores in advance a parking position where the vehicle parks, determines whether there is a visitor visiting a building located around the parking position based on the surrounding information including the situation around the vehicle parked at the parking position, and when it is determined that there is a visitor, notifies notification information regarding the visitor to a user of the vehicle or a person related to the building.

Images