Monitoring method and monitoring device

Abstract

To provide a monitoring method and a monitoring device to realize smooth communication from outside with a person inside a vehicle.SOLUTION: A monitoring device 100 monitors surroundings of a vehicle 1. The monitor device 100 includes: a detection unit 131 for detecting a moving object existing around the vehicle 1 based on surrounding information around the vehicle 1; and a device control unit 133 for making a predetermined device to execute a notification process for notifying a person inside the vehicle 1 of information indicating the position of the moving object with respect to the vehicle 1 when the moving object is detected while the vehicle 1 is parked.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a monitoring method and a monitoring device for monitoring the surroundings of a vehicle. [Background technology]

[0002] In recent years, it has become common for people to use the interior space of a parked vehicle as personal space. For example, it may be used as a space for hobbies such as watching movies, watching television, or reading, or as a work space for computer work or meetings. Technologies have been proposed for making effective use of such interior space of a vehicle. For example, a technology has been proposed in which an experience device is operated in a predetermined mode inside the vehicle when the user is not driving, providing the user with a comfortable experience (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-189618 Summary of the Invention [Problem to be solved by the invention]

[0004] While the above-described conventional technology can realize an environment in which experience devices can be utilized within the interior space of a vehicle, it often also creates an environment that blocks sounds from outside the vehicle. In such a case, for example, even if a person inside the vehicle approaches from outside the vehicle, it is conceivable that the person inside the vehicle may be slow to notice the approaching person. In such an environment, for example, it may be difficult to smoothly communicate between the person inside the vehicle and the person outside the vehicle.

[0005] An object of the present invention is to realize smooth communication from outside with people inside a vehicle. [Means for solving the problem]

[0006] One aspect of the present invention is to By computer This monitoring method includes a detection step of detecting a moving object existing around a vehicle based on surrounding information about the vehicle, and a control step of causing a predetermined device to execute a notification process for notifying a person inside the vehicle of information indicating the position of the moving object relative to the vehicle when the moving object is detected while the vehicle is parked. Let the computer run In the control step, the notification method for the notification process when it is determined that the detected moving object is moving toward the center of the vehicle is different from the notification method for the notification process when it is determined that the moving object has stayed within a specified range based on the vehicle for a specified period of time or more. [Effects of the Invention]

[0007] According to the present invention, smooth communication from outside with people inside a vehicle can be realized. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a simplified diagram showing the relationship between a parked vehicle and moving objects existing around the vehicle. [Figure 2] FIG. 2 is a top view showing an example of the exterior configuration of a vehicle. [Figure 3] FIG. 3 is a block diagram showing an example of the functional configuration of a monitoring device mounted on a vehicle. [Figure 4] FIG. 4 is a simplified diagram showing an example of a determination process for determining a moving object existing around a vehicle using a sonar. [Figure 5] FIG. 5 is a simplified diagram showing an example of a determination process for determining a moving object existing around a vehicle using a sonar. [Figure 6] FIG. 6 is a simplified diagram showing the interior space of a vehicle. [Figure 7] FIG. 7 is a simplified diagram showing a vehicle and people detected in its surroundings. [Figure 8]FIG. 8 is a simplified diagram showing a vehicle and people detected in its surroundings. [Figure 9] FIG. 9 is a flowchart illustrating an example of a notification process in the monitoring device. [Figure 10] FIG. 10 is a flowchart showing an example of a notification process in the monitoring device. [Figure 11] FIG. 11 is a simplified diagram showing the relationship between a parked vehicle and an air vehicle existing around the vehicle. [Figure 12] FIG. 12 is a simplified diagram showing a vehicle and detected flying objects around the vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0010] [Example of the relationship between a vehicle and surrounding moving objects] Fig. 1 is a simplified diagram showing the relationship between a parked vehicle 1 and moving objects existing around the vehicle 1. In Fig. 1, an explanation will be given using people U1 to U5 as an example of moving objects.

[0011] Person U1 is approaching vehicle 1 to meet person U21 (see FIG. 6) inside vehicle 1, as indicated by arrow AR1. Furthermore, persons U2 to U5 are people passing around vehicle 1 unrelated to person U21 inside vehicle 1, as indicated by arrows AR2 to AR5. In other words, persons U2 to U5 are examples of people who simply pass by vehicle 1. Person U5 is assumed to be riding a bicycle.

[0012] It is also assumed that the person U21 uses the interior space of the parked vehicle 1 as a personal space. For example, it is assumed that the space may be used for hobbies such as watching movies, watching television, or reading, or for work such as computer work or meetings. When engaging in these hobbies or work, the person U21 may block the view from around the vehicle 1 by blocking the side windows 15 to 18, or may block external sounds by using electronic devices ED1 (see FIG. 6) such as headphones. In this manner, in an environment where the view from outside the vehicle 1 is blocked or sounds from outside the vehicle 1 are blocked, information from outside the vehicle 1 can be shut out. However, for example, even if a person U1 who has business with the person U21 inside the vehicle approaches the person U21 from outside the vehicle 1, it is assumed that the person U21 inside the vehicle may be slow to notice the approaching person U1. In other words, it is assumed that communication from outside the vehicle 1 may be difficult.

[0013] In such a case, when person U1 outside the vehicle attempts to communicate with person U21, person U1 must confirm whether person U21 is in vehicle 1. Furthermore, person U1 must take actions such as peering into the interior of vehicle 1, where line of sight is blocked, or knocking on vehicle 1 to call out to person U21 inside. In this case, person U1 must perform complicated actions before communicating with person U21, which increases the time it takes for communication to occur. Furthermore, person U21 may take a long time to notice person U1's presence, even though person U1 is approaching, and it may also take a long time for communication to occur. Alternatively, person U21 may not notice person U1's presence and may not be able to meet person U1, which could result in a loss of an opportunity to communicate with person U1.

[0014] Therefore, in this embodiment, various sensors (for example, ultrasonic sensors, image sensors, sound acquisition sensors) mounted on the vehicle 1, electronic devices capable of communicating with the vehicle 1, and the like are utilized to enable smooth communication between people inside and outside the vehicle 1. Specifically, a person U21 inside the vehicle 1 and a person U1 who wants to communicate are determined, and a predetermined notification process is executed for the person U21 inside the vehicle and the person U1 outside the vehicle.

[0015] That is, even in an environment where the view around the vehicle 1 is blocked or sounds from outside the vehicle 1 are blocked while the vehicle 1 is parked, smooth communication can be carried out between the person U21 inside the vehicle and the person U1 outside the vehicle. In other words, the ease of communication can be improved for the parked vehicle 1.

[0016] [Example of vehicle exterior configuration] 2 is a top view showing an example of the exterior configuration of the vehicle 1. The vehicle 1 is a vehicle such as an internal combustion engine vehicle, a hybrid vehicle, or an electric vehicle.

[0017] Vehicle 1 includes a roof 2, a front window 3, a rear window 4, doors 11 to 14, side windows 15 to 18, a front wiper 21, a rear wiper 22, and wing mirrors 31 and 32. The vehicle 1 also includes a plurality of light-emitting units in an upper portion of the interior thereof. In FIG. 2, the positions of the light-emitting units installed in the interior of vehicle 1 are indicated by dotted circles 41 to 47. In FIG. 6, light-emitting units 6 to 7 corresponding to dotted circles 43 to 45 are shown. These light-emitting units are realized by light-emitting elements such as LEDs (light emitting diodes). At least one of the opening and closing of doors 11 to 14, locking and unlocking of doors 11 to 14, opening and closing of side windows 15 to 18, operation of front wiper 21, operation of rear wiper 22, and operation of wing mirrors 31 and 32 is controlled by a control unit 130 (see FIG. 2).

[0018] [Example of functional configuration of monitoring device] FIG. 3 is a block diagram showing an example of the functional configuration of the monitoring device 100 mounted on the vehicle 1. As shown in FIG.

[0019] The monitoring device 100 includes sensors 111, a sound acquisition unit 112, an image acquisition unit 113, a communication unit 120, a control unit 130, a memory unit 140, a light-emitting unit group 151, an interior sound output unit 152, an exterior sound output unit 153, an interior display unit 154, an exterior display unit 155, and an equipment driving unit group 156. The control unit 130 also includes a detection unit 131, a determination unit 132, and an equipment control unit 133.

[0020] The sensors 111 are various sensors installed on the vehicle 1 to detect moving objects existing around the vehicle 1, and output the detection values ​​to the detection unit 131. The sensors 111 are, for example, LIDAR (Light Detection and Ranging) and sonar. Note that these are merely examples, and other sensors may also be used.

[0021] The sound acquisition unit 112 is provided inside or outside the vehicle 1, and acquires sounds around the vehicle 1 (sounds outside the vehicle) based on the control of the control unit 130, and outputs information about the acquired sounds to the detection unit 131 and the determination unit 132. As the sound acquisition unit 112, for example, one or more microphones or sound acquisition sensors can be used.

[0022] The image acquisition unit 113 is provided inside or outside the vehicle 1, and captures an image of a subject to generate an image (image data) under the control of the control unit 130, and outputs information about the generated image to the detection unit 131 and the determination unit 132. The image acquisition unit 113 is configured, for example, with one or more camera devices or image sensors that can capture images of subjects around the vehicle 1. That is, the image acquisition unit 113 can acquire images of subjects that exist in the left-right and front-rear directions of the vehicle 1, or in all directions. Note that the image acquisition unit 113 may also acquire images of the sky above the vehicle 1.

[0023] The communication unit 120 exchanges various types of information with other devices using wireless communication under the control of the device control unit 133. For example, the communication unit 120 receives signals from a key for the vehicle 1 (e.g., an intelligent key) and signals from electronic devices (e.g., electronic devices ED1 and ED2 shown in FIG. 6 and electronic device ED3 shown in FIG. 7) using short-range wireless communication (e.g., wireless LAN such as Wi-Fi, Bluetooth (registered trademark), and wireless communication using long waves or ultra-high frequency waves). In addition, for example, the communication unit 120 acquires various types of information from various devices via a predetermined network (e.g., the Internet).

[0024] The control unit 130 controls each unit based on various programs stored in the storage unit 140. The control unit 130 is realized by a processing device such as a CPU (Central Processing Unit). Note that the vehicle ECU (Electronic Control Unit) of the vehicle 1 may also be used as the control unit 130, or a processing device different from the vehicle ECU may be provided as the control unit 130.

[0025] The detection unit 131 detects a moving object present around the vehicle 1 based on information (surrounding information of the vehicle 1) output from at least one of the sensors 111, the sound acquisition unit 112, the image acquisition unit 113, and the communication unit 120. The detection unit 131 then outputs the detection result to the determination unit 132. The method for detecting a moving object will be described in detail with reference to Figs. 4, 5, etc.

[0026] When the detection unit 131 detects a moving object while the vehicle 1 is parked, the determination unit 132 determines whether the moving object is related to a person U21 (see FIG. 6) inside the vehicle 1, and outputs the determination result to the device control unit 133. Specifically, the determination unit 132 performs a determination process based on the detection information output from the detection unit 131 and information (surrounding information of the vehicle 1) output from at least one of the sensors 111, the sound acquisition unit 112, the image acquisition unit 113, and the communication unit 120. For example, the determination unit 132 performs the determination process based on at least one of the degree of agreement between the information about the moving object detected by the detection unit 131 and the registered information, and the behavioral characteristics of the moving object. Note that a determination method using the behavioral characteristics of the moving object will be described in detail with reference to FIGS. 4, 5, etc. Furthermore, the degree of agreement between the information about the moving object and the registered information will be described in detail with reference to FIG. 9, etc.

[0027] When a moving object is detected by the detection unit 131 while the vehicle 1 is parked, the device control unit 133 causes a predetermined device to execute a notification process for notifying a person U21 inside the vehicle 1 of information indicating the position of the moving object relative to the vehicle 1. Specifically, when the detection unit 131 detects a moving object and the determination unit 132 determines that the moving object is related to the person U21 inside the vehicle 1, the device control unit 133 causes the predetermined device to execute the notification process. The predetermined device is, for example, a light-emitting unit group 151, an in-vehicle sound output unit 152, an outside-vehicle sound output unit 153, an in-vehicle display unit 154, an outside-vehicle display unit 155, a device driving unit group 156, electronic devices E1 to ED3 (see FIGS. 6 to 8), and a vibration device (not shown).

[0028] In addition to performing this notification process, the device control unit 133 also performs an external notification process to notify the outside of the vehicle 1 of information indicating that a moving object related to the person U21 inside the vehicle 1 has been detected and that the person U21 inside the vehicle 1 has been notified. For example, as shown in Figures 7 and 8, the external notification process can be performed by audio output of predetermined messages M1 and M2. In this way, the device control unit 133 functions as a notification unit that notifies the inside and outside of the vehicle 1 of information indicating at least one of the direction of the moving object detected by the detection unit 131 and the distance from the vehicle 1 to the moving object using a predetermined notification method.

[0029] The storage unit 140 is a storage medium that stores various types of information. For example, the storage unit 140 stores various types of information (e.g., control programs and registration information) required for the control unit 130 to perform various processes. The storage unit 140 also stores various types of information acquired via the communication unit 120. As the storage unit 140, various types of storage media such as a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a solid state drive (SSD) can be used. The registration information is information used when determining whether a moving object detected by the detection unit 131 is related to a person inside the vehicle 1. As the registration information, for example, image identification information for identifying an image of the moving object, sound identification information for identifying a sound emitted by the moving object, and device identification information for identifying an electronic device carried by the moving object are stored in the storage unit 140. The registration information may be stored in an external device and acquired and used via the communication unit 120 as needed.

[0030] The light-emitting unit group 151 is made up of various light-emitting units installed in the vehicle 1, and the light-emitting state is controlled based on the control of the device control unit 133. The light-emitting unit group 151 is, for example, light-emitting units installed inside the vehicle 1 (interior lights installed at the positions of dotted circles 41 to 47 shown in FIG. 2) and light-emitting units installed outside the vehicle 1 (for example, turn signals, brake lights, and headlights). Note that these are merely examples, and other light-emitting units may also be used.

[0031] The interior sound output unit 152 outputs sound to the interior of the vehicle 1 based on the control of the device control unit 133. As the interior sound output unit 152, for example, one or more speakers installed inside the vehicle 1 can be used.

[0032] The vehicle exterior sound output unit 153 outputs sound to the outside of the vehicle 1 based on the control of the device control unit 133. As the vehicle exterior sound output unit 153, for example, one or more speakers installed outside the vehicle 1 can be used.

[0033] The interior display unit 154 is a display unit installed inside the vehicle 1, and its display state is controlled based on the control of the device control unit 133. The interior display unit 154 may be, for example, one or more display panels installed inside the vehicle 1. Images may also be displayed on the ceiling (roof 2), front window 3, rear window 4, side windows 15 to 18, etc. of the vehicle 1 as display media. For example, various images may be displayed on the ceiling of the vehicle 1 using a display device (e.g., a projector, an optical system). When various images are displayed on the front window 3, rear window 4, and side windows 15 to 18, various materials that can be switched between a light-blocking state and a light-transmitting state may be used for each window. For example, photochromic materials, electrochromic light-control glass, translucent liquid crystal, organic electroluminescence (EL), gasochromic light-control sheets, etc. may be used. Various images may be displayed on each window using a display device (e.g., a projector, an optical system). In addition, each window itself may be configured as a transparent display panel, and various images may be displayed on each window. In addition, if the windshield 3 functions as a display medium for a HUD (Head Up Display) of the vehicle 1, the HUD display can be displayed on the windshield 3.

[0034] The exterior display unit 155 is a display unit installed outside the vehicle 1, and the display state is controlled based on the control of the device control unit 133. As the exterior display unit 155, for example, one or more display panels installed outside the vehicle 1 can be used. Furthermore, the roof 2, the front window 3, the rear window 4, the doors 11 to 14, the side windows 15 to 18, etc. may be used as display media to display each image. Alternatively, display panels may be installed on these surfaces to display each image.

[0035] The equipment driving unit group 156 drives movable equipment among the equipment installed in the vehicle 1, and the driving state of each equipment is controlled based on the control of the equipment control unit 133. The equipment driven by the equipment driving unit group 156 includes, for example, the doors 11 to 14, the side windows 15 to 18, the front wiper 21, the rear wiper 22, and the wing mirrors 31 and 32.

[0036] [Example of detecting moving objects using sonar] 4 and 5 are simplified diagrams showing an example of a determination process using sonar to determine a moving object existing around the vehicle 1. The sonar is an example of the sensors 111.

[0037] As shown in Figures 4 and 5, ultrasonic sensors 51 to 60 that output sound waves are installed on vehicle 1. Note that in Figures 4 and 5, the portions of vehicle 1 where ultrasonic sensors 51 to 60 are installed are indicated by dotted circles. Also, although Figures 4 and 5 only show the front side of vehicle 1, it is assumed that ultrasonic sensors are also installed on the rear side of vehicle 1. Note that ultrasonic sensors 51 to 60 are sensors that emit ultrasonic waves (transmitted waves) and measure the presence or distance of an object using reflected waves (received waves) reflected by the object.

[0038] Fig. 4 shows an example of determining whether a person U10 is moving toward the center of the vehicle 1 (in the direction of arrows AR10 and AR11). Fig. 4 also shows an example of a transition when the person U10 moves from a left-side state to a right-side state. The dotted arrows indicate distances L1 to L6 to the person U10 detected by the ultrasonic sensors 55 to 57.

[0039] When an object whose detection distance L is less than the threshold value TH1 is detected by at least one of the ultrasonic sensors 51 to 60, the detection unit 131 detects the object using the detection values ​​of the ultrasonic sensor that detected the object and the surrounding ultrasonic sensors. In the example shown on the left side of FIG. 4, the shortest detection distance L2 for the person U10 and the distances L1 and L3 to the person U10 detected by the ultrasonic sensors 55 and 57 on the left and right sides are assumed to be less than the threshold value TH1. In this case, the detection unit 131 can estimate the position of the person U10 using a triangle determined by the distance between the ultrasonic sensors 55 and 57 and the distances L1 and L3. Similarly, in the example shown on the right side of FIG. 4, the shortest detection distance L5 for the person U10 and the distances L4 and L6 to the person U10 detected by the ultrasonic sensors 55 and 57 on the left and right sides are assumed to be less than the threshold value TH1. That is, the detection unit 131 estimates the position of the person U10 based on a triangle determined by the distance between the ultrasonic sensors 55 and 57 and the distances L4 and L6.

[0040] The detection unit 131 also determines the moving direction of the person U10 based on the detection values ​​(distances L1 to L6) output from the ultrasonic sensors 55 to 57. Specifically, the detection unit 131 determines the moving direction of the person U10 based on the rate of change between the detection values ​​(distances L1 to L3) in the state shown on the left side of FIG. 4 and the detection values ​​(distances L4 to L6) in the state shown on the right side of FIG. 4. That is, the moving direction of the person U10 relative to the vehicle 1 can be determined based on the rate of change between the distance L2, L5 with the shortest detection value and the distances L1, L3, L4, and L6 to the left and right of the distance L2, L5. Alternatively, the detection unit 131 can determine the moving direction of the person U10 based on the transition of the position of the person U10 estimated based on the distance between the ultrasonic sensors 55 and 57, the distances L1 and L3, and the distances L4 and L6.

[0041] Fig. 5 shows an example of determining whether a person U11 has been staying around the vehicle 1 for a long period of time. Fig. 5 also shows an example of a transition when the person U11 moves from a left-side state to a right-side state. Note that the dotted arrows indicate the distances L7 to L9 to the person U11 detected by the ultrasonic sensors 57 and 58.

[0042] In the example shown in Fig. 5, it is assumed that distances L7 to L9 to person U11 are less than threshold TH1. In this case, in the example shown on the left side of Fig. 5, detection unit 131 measures the position of person U10 relative to vehicle 1 using distance L7 to person U11. Similarly, in the example shown on the right side of Fig. 5, detection unit 131 measures the position of person U11 relative to vehicle 1 using distances L8 and L9 to person U11. Then, detection unit 131 measures the position of person U11 relative to vehicle 1 using distances L7 to L9 to person U11, and determines the direction of movement of person U11 based on changes in that position.

[0043] Specifically, the detection unit 131 determines the moving direction of the person U11 based on the rate of change between the detection value (distance L7) in the state shown on the left side of FIG. 5 and the detection value (distances L8, L9) in the state shown on the right side of FIG. 5.

[0044] In the example shown in FIG. 5, the person U11 is not moving toward the center of the vehicle 1 but is moving around the vehicle 1. For example, the person U11 may have business with a person U21 inside the vehicle 1, but may be wandering around the vehicle 1 searching for the person U21 inside the vehicle 1 because the person U21's presence is unknown. In this case, whether the person U11 is a person (object) who has business with the person U21 inside the vehicle 1 can be determined based on the position of the person U11 relative to the vehicle 1 and the time the person U11 is present around the vehicle 1. Specifically, when an object not moving toward the center of the vehicle 1 is detected, the object can be determined based on whether the object has remained within a predetermined range R1 from the vehicle 1 for a predetermined time T1 or more. The predetermined range R1 is a range that can be detected by the ultrasonic sensors 51 to 60 and may be, for example, a range of several meters from the vehicle 1. The predetermined time T1 is the time during which a person who has business inside the vehicle 1 is expected to stay, and can be set appropriately based on experimental data, etc. For example, the predetermined time T1 can be set to several seconds to several tens of seconds.

[0045] This makes it possible to distinguish between people who are simply passing by the vehicle 1, such as the people U2 to U5 shown in FIG. 1, and a person U11 who has business with the person U21 in the vehicle 1.

[0046] [Other examples of motion detection] 4 and 5 show a detection method using sonar to detect a moving object, but other detection methods may also be used to detect a moving object. For example, a detection method using other sensors such as LIDAR to detect an object may also be used.

[0047] Alternatively, for example, a detection method may be used that detects an object included in an image based on the image acquired by the image acquisition unit 113. This detection method may use a known object detection technology. This object detection technology may be a technology for detecting each object, such as a human body detection technology or a face detection technology.

[0048] Also, for example, a detection method may be used that detects objects around the vehicle 1 based on sounds acquired by the sound acquisition unit 112. For example, based on changes in the sounds acquired by the sound acquisition unit 112, it is possible to estimate the movement state of the object emitting the sound toward the vehicle 1. Also, other known object detection techniques using sound may be used.

[0049] Alternatively, for example, a detection method may be used in which an electronic device capable of wireless communication with the communication unit 120 is used to detect objects around the vehicle 1. For example, when the communication unit 120 receives radio waves from an electronic device capable of wireless communication, the distance between the electronic device and the vehicle 1 can be estimated based on the radio wave intensity (reception intensity). That is, the distance from the person carrying the electronic device or the vehicle carrying the electronic device to the vehicle 1 can be estimated. Furthermore, for example, when the communication unit 120 communicates using multiple antennas, the direction of the radio waves emitted from the electronic device can be estimated. That is, the direction from the vehicle 1 of the person carrying the electronic device or the vehicle carrying the electronic device can be detected. In this way, the radio waves emitted from the electronic device can be used to detect the position (distance and direction) of the electronic device relative to the vehicle 1, thereby making it possible to detect the position of a moving object (e.g., a person or a vehicle) carrying the electronic device. Other well-known object detection techniques using radio waves may also be used.

[0050] Alternatively, a detection method may be used in which an object around the vehicle 1 is detected using a location information acquisition unit provided in the vehicle 1 and a location information acquisition unit provided in an electronic device capable of wireless communication. The location information acquisition unit may be realized, for example, by a GNSS receiver that acquires location information using a Global Navigation Satellite System (GNSS). The location information includes data related to the position, such as latitude, longitude, and altitude, at the time of receiving the GNSS signal. The location information may also be acquired using other location information acquisition methods. For example, the location information may be derived using information from surrounding access points or base stations. Alternatively, the location information may be derived using a position estimation technique using a navigation device. For example, the detection unit 131 acquires, via the communication unit 120, the location information of the electronic device acquired by the location information acquisition unit provided in the electronic device. The detection unit 131 can then detect the position (distance and direction) of the electronic device relative to the vehicle 1 based on the location information of the vehicle 1 acquired by the location information acquisition unit provided in the vehicle 1 and the location information of the electronic device. This allows the detection unit 131 to detect the position of a moving object (for example, a person or a vehicle) carrying the electronic device.

[0051] [Example of using other criteria together with motion detection] The person U10 and the person U11 detected by the detection methods shown in Figures 4 and 5 can be determined to be a person (moving object) who has business with the person U21 inside the vehicle 1. However, by combining other determination conditions with these determinations, the accuracy of the determination can be improved. Therefore, below, a determination method for determining a moving object who has business with the person U21 inside the vehicle 1 by combining multiple determination conditions will be described.

[0052] For example, together with the moving object detection shown in Figures 4 and 5, information about the moving object detected by the moving object detection and the degree of coincidence of the registration information can be used to determine the person (object) that person U21 inside vehicle 1 has business with.

[0053] Here, the registration information is information that has been registered in advance by a user of the vehicle 1 (e.g., person U21), and is stored in the storage unit 140. As described above, the registration information includes image identification information, sound identification information, device identification information, etc. Note that the sound identification information is information about sounds emitted by people, animals, and devices, such as information for identifying voiceprints and information for identifying voice patterns.

[0054] The image identification information is, for example, information for identifying a human face as a moving object. For example, a face identification method may be used that identifies a face included in an image based on the image acquired by the image acquisition unit 113. This face identification method may use a known face identification method. For example, a recognition technology for each object, such as an individual recognition technology or a face recognition technology, may be used as this face identification method.

[0055] The electronic devices registered in the device identification information are devices that can cooperate with the vehicle 1. For example, they are devices that can exchange information with the communication unit 120 using wireless communication, and are devices that can be carried by a person or mounted on a vehicle. These electronic devices are, for example, devices such as a key for the vehicle 1 (e.g., an intelligent key), a mobile terminal (e.g., a mobile phone, a smartphone, or a tablet device), or a wearable terminal (e.g., a smart watch). The device identification information is unique identification information (e.g., device-specific information, terminal identification information) assigned to the electronic device, such as a MAC (Media Access Control) address or a user ID. For example, in order to exchange various types of information with the vehicle 1, the identification information of the electronic device can be stored in the storage unit 140 as registration information (e.g., user registration), thereby registering the electronic device.

[0056] A voiceprint is information that indicates the characteristics of a voice that are determined by analyzing the voice of an individual. For example, the voiceprints of a person U21's family members and associates can be registered.

[0057] Voice pattern recognition is information that indicates specific words or sentences uttered by a person. For example, words or sentences such as "Excuse me," "Is anyone there?", "Did you hear me?", "Um...," "Dad," and "Dinner is ready" can be registered as voice pattern recognition. It is also possible to register sounds of moving objects, such as a knock on the door of the vehicle 1, and sounds emitted by various devices.

[0058] For example, if the information about the moving object detected by the moving object detection shown in Figures 4 and 5 matches the registration information, it is possible to determine whether the person U21 inside the vehicle 1 has business with them (object). This determination example will be described in detail with reference to Figure 9.

[0059] In this way, in this embodiment, people who want to communicate with person U21 can be determined using the movement characteristics of moving objects around vehicle 1 (for example, how a person approaches), as well as registration information such as voice and owned devices.

[0060] Here, if it is possible to perform individual identification processing on a moving object included in an image acquired by the image acquisition unit 113, part of the moving object detection shown in Figures 4 and 5 may be omitted. This example will be described in detail with reference to Figure 10.

[0061] Furthermore, when moving object detection is performed using radio waves obtainable via the communication unit 120, it is possible to determine whether or not the electronic device is registered using the identification information of the electronic device contained in the received radio waves. Therefore, when moving object detection is performed using radio waves, it is also possible to perform moving object determination using the radio waves.

[0062] Furthermore, if it is possible to perform facial expression identification processing on the facial expressions of people included in the images acquired by the image acquisition unit 113, the facial expressions can be used to determine who wants to communicate with the person U21. For example, if the facial expression is one that suggests the person wants to talk to the vehicle 1, the person can be determined to be a person who wants to communicate with the person U21.

[0063] [Example of notification to a person inside a vehicle] FIG. 6 is a simplified diagram showing the interior space 5 of the vehicle 1. FIG. 6 simply shows only the interior space 5 at the rear of the vehicle 1. Light-emitting units 6 to 8 are installed at the top of the interior space 5 of the vehicle 1. The light-emitting units 6 to 8 correspond to the dotted circles 43 to 45 shown in FIG. 2. A display panel or a speaker may be installed instead of the light-emitting units 6 to 8 to provide notification, or a speaker may be built into or attached to the light-emitting units 6 to 8 and used for notification.

[0064] 6 shows a person U21 seated in the rear of the interior space 5 of the vehicle 1 engaging in hobbies or work using electronic devices ED1 and ED2. The electronic device ED1 is, for example, a sound output device (e.g., wireless headphones) that outputs sound using wired or wireless communication. The electronic device ED2 is, for example, an information processing device (e.g., a smartphone or tablet device) that outputs images and sound using wired or wireless communication.

[0065] As shown in Figure 6, if person U21 sitting at the rear of interior space 5 of vehicle 1 is concentrating on something, it is expected that it will be difficult for person U21 to recognize the presence of someone outside vehicle 1 approaching person U21 who has business with him or her.

[0066] Therefore, in this embodiment, each device installed in the vehicle 1 is used to notify the person U21 that there is a person outside the vehicle 1 who is presumed to have business with the person U21. Specifically, as shown in FIG. 1 , when a person U1 moving toward the center of the vehicle 1 is detected, the light-emitting unit 7 corresponding to the direction in which the person U1 was detected is turned on or blinked to notify the person U21 that the person U1 has been detected. This allows the person U21 to easily understand that a person U1 who is presumed to have business with the person U21 has come near the vehicle 1.

[0067] In addition to notifying the direction of the moving object, the distance between the vehicle 1 and the moving object may also be notified. For example, the distance between the vehicle 1 and the moving object can be notified by varying the intensity of the light from the light-emitting unit 7. For example, when the distance between the vehicle 1 and the person U1 is long, the light-emitting unit 7 is made to emit a weak light, and as the distance between the vehicle 1 and the person U1 decreases, the light-emitting unit 7 is made to emit a stronger light. Also, for example, the distance between the vehicle 1 and the moving object can be notified by varying the blinking interval of the light from the light-emitting unit 7. For example, when the distance between the vehicle 1 and the person U1 is long, the light-emitting unit 7 is made to emit a longer blinking interval, and as the distance between the vehicle 1 and the person U1 decreases, the light-emitting unit 7 is made to emit a shorter blinking interval. Also, for example, the distance between the vehicle 1 and the moving object can be notified by varying the color of the light from the light-emitting unit 7. For example, when the distance between the vehicle 1 and the person U1 is long, the light-emitting unit 7 is made to emit a color similar to blue or green, and as the distance between the vehicle 1 and the person U1 decreases, the light-emitting unit 7 is made to emit a color ranging from yellow to red.

[0068] Notification may also be made using a device other than the light-emitting unit 7. For example, the device control unit 133 may cause the in-vehicle sound output unit 152 to output a voice message such as, "A related person is approaching from the rear left. He is 3 m from the vehicle." For example, the device control unit 133 may control the electronic device ED1 via the communication unit 120 to output a voice message such as, "A related person is approaching from the rear left. He is 3 m from the vehicle." The device control unit 133 may also cause the electronic device ED1 to output a voice message such as, "A related person is approaching from the rear left. He is 3 m from the vehicle." The device control unit 133 may also cause the electronic device ED2 ... display information such as the direction and distance of the person U1 on the display unit of the electronic device ED2.

[0069] Notification may also be made using a vibration device that performs a predetermined vibration operation. For example, vibration devices are installed at positions corresponding to six light-emitting units (dotted circles 41 to 46 shown in FIG. 2), and the vibration device installed at the position corresponding to the light-emitting unit to be illuminated is vibrated. This allows the person U21 to quickly execute notification by receiving a physical notification from the vibration device as well as a visual notification from the light-emitting unit. Furthermore, similar to the light-emitting unit, the distance between the vehicle 1 and the moving object may be notified by varying the strength of the vibration of the vibration device.

[0070] Furthermore, the notification may be provided using at least one of the side windows 15 to 18, the front wiper 21, the rear wiper 22, and the wing mirrors 31 and 32. For example, the side windows 15 to 18 may be assigned to four light-emitting elements (dotted circles 42, 43, 45, and 46 shown in FIG. 2), and the front wiper 21 and the rear wiper 22 may be assigned to two light-emitting elements (dotted circles 41 and 44 shown in FIG. 2). The notification may be provided using a device located at a position corresponding to the light-emitting element to be illuminated. For example, when the light-emitting element 7 is illuminated, the corresponding side window 17 is opened. This example is shown in FIG. 8. When the light-emitting element 8 is illuminated, the corresponding rear wiper 22 is operated. Furthermore, when a person approaching from the left or right in front of the vehicle 1 is detected, the wing mirrors 31 and 32 may be opened or closed.

[0071] Also, instead of opening and closing the side windows 15 to 18, the doors 11 to 14 corresponding to the direction of the person approaching the vehicle 1 may be opened, closed, or unlocked. For example, if the door corresponding to the direction of the person approaching the vehicle 1 is a sliding door, the sliding door may be opened, and if the corresponding door is not a sliding door, the door may be unlocked. This allows the person U21 inside the vehicle 1 to quickly get out and meet the person U1.

[0072] Furthermore, if curtains are installed inside the vehicle 1, the curtains corresponding to the direction of a person approaching the vehicle 1 may be opened.

[0073] The notification may also be made using other devices installed inside the vehicle 1. For example, if a car navigation device or an audio device is installed inside the vehicle 1, the notification information may be output from these devices. For example, the notification may be made by display on a display panel of the car navigation device, audio output, or audio output from the audio device.

[0074] These notification methods are examples of how to provide a predetermined notification to a person U21 inside the vehicle 1, and the predetermined notification may be provided by other notification methods. Furthermore, the above-mentioned notification methods may be combined as appropriate, taking into consideration the preferences of the person inside the vehicle 1 and the effect of the notification on that person.

[0075] [Example of changing notification method depending on the position and situation of people in the vehicle] Furthermore, the notification method may be changed depending on the position of the person U21 inside the vehicle 1 and the situation of the person U21. The position of the person U21 inside the vehicle 1 and the situation of the person U21 can be estimated by an image acquisition unit (e.g., a camera) or a sound acquisition unit (e.g., a microphone) installed inside the vehicle 1.

[0076] For example, if the person U21 is sleeping inside the vehicle 1, it is preferable to use a notification method using audio or vibration rather than notification using an image display. Also, for example, if the person U21 is looking at a display panel at the front inside the vehicle 1, it is preferable to use a notification method using an image display on the display panel. Also, for example, if the person U21 is using a sound output device (e.g., wireless headphones) at the rear inside the vehicle 1, it is preferable to use a visual notification method or a notification method using vibration. In this case, it is expected that it will be difficult for the person U21 to see the display panel at the front, so it is preferable to use another display method. For example, a vibration device located closest to the person U21 inside the vehicle 1 may be vibrated, and the direction of the person U1 approaching the vehicle 1 may be notified by emitting light from a light-emitting unit.

[0077] Furthermore, for example, if all the light-emitting units inside the vehicle 1 are lit, it may be impossible to notify by lighting up the light-emitting units. In this case, it is preferable to use a notification method such as sound or vibration. Alternatively, the direction of a person approaching the vehicle 1 may be notified by flashing the light-emitting units or by displaying a color different from the color of the other light-emitting units.

[0078] [Example of changing notification method depending on the characteristics of the movement of moving objects around the vehicle] The notification method may be changed depending on the characteristics of the movement of a moving object detected around the vehicle 1. For example, as shown in FIG. 4, the movement of a person U10 moving toward the center of the vehicle 1 is presumed to be clearly exhibiting a behavioral pattern of attempting to communicate with a person U21 inside the vehicle 1. On the other hand, as shown in FIG. 5, the movement of a person U11 who stays within a predetermined range R1 from the vehicle 1 for a predetermined time T1 or more does not clearly exhibit a behavioral pattern of attempting to communicate with the person U21 inside the vehicle 1. However, the behavior of the person U11 can also be presumed to be an attempt to communicate with the person U21 in some way. For example, it is possible that the environment around the parked vehicle 1 changes (e.g., from day to night, or from an empty state to a crowded state). When the environment around the parked vehicle 1 changes in this way, it is possible that people will gather around the vehicle 1 without anyone noticing. In this case, considering security reasons, it is important to notify the person U21 inside the vehicle 1 of the change in the environment around the vehicle 1, even if there is no one presumed to be clearly communicating with the person U21.

[0079] Therefore, for example, as shown in FIG. 4, when a person U10 moving toward the center of the vehicle 1 is detected, it is preferable to use a notification method that clearly notifies the direction in which the person U10 is located. For example, notification can be made by lighting or flashing a light-emitting element corresponding to the direction in which the person U10 is located. On the other hand, as shown in FIG. 5, when a person U11 is detected who has been within a predetermined range R1 from the vehicle 1 for a predetermined time T1 or more, it is preferable to notify a person U21 inside the vehicle 1 that the environment around the vehicle 1 has changed, including for security purposes. For example, all light-emitting elements inside the vehicle 1 can be made to light up, all flash, or all can be made to have a specific color (e.g., yellow). However, even in this case, the direction and distance of the person U11 approaching the vehicle 1 can also be notified. For example, when all light-emitting elements inside the vehicle 1 are made to light up, only the light-emitting element corresponding to the direction in which the person U11 is located can be made to flash. Furthermore, when the color of all light-emitting elements inside the vehicle 1 is made to have a specific color (e.g., yellow), after a predetermined time has elapsed, only the light-emitting element corresponding to the direction in which the person U11 is located can be made to have a different color (e.g., red).

[0080] [Example 1 of notification to people around the vehicle] 7 is a simplified diagram showing the vehicle 1 and a person U1 detected around the vehicle 1. It is assumed that the person U1 is a person moving in the direction of the arrow AR1 shown in FIG.

[0081] 7, even if a person U21 is in the vehicle 1, it is assumed that the person U21 may not be visible from the outside due to, for example, blackout glass, curtains, etc. In this case, it is difficult to determine whether the person U21 is in the vehicle 1, and it is assumed that it may take a long time for the person U1 and the person U21 to meet and talk.

[0082] Therefore, in this embodiment, each device installed in the vehicle 1 is used to notify people outside the vehicle 1 who are presumed to have business with the person U21 that the person U21 is present. For example, as shown in FIG. 7, a predetermined message M1 is output as audio from the vehicle exterior sound output unit 153. In this case, the audio output may be directed in the direction in which the person U1 is present. This allows the person U1 to easily understand that the person U21 will emerge from the vehicle 1 if they wait a short while. The position of the person U21 inside the vehicle 1 may also be notified. For example, the person U1 may be notified that the person U21 is sitting on the left side of the rear section. This allows the person U1 to easily understand the position in which the person U21 will emerge from the vehicle 1, and to quickly meet the person U21.

[0083] [Example 2 of notification to people around the vehicle] Fig. 8 is a simplified diagram showing a vehicle 1 and a person U1 detected around the vehicle 1. The example shown in Fig. 8 is substantially the same as Fig. 7 except that the side window 17 is opened to notify the person U1 outside the vehicle 1. Therefore, the following description will focus on the differences from Fig. 7.

[0084] 8, a predetermined message M2 is outputted by voice from the vehicle exterior sound output unit 153, and the device control unit 133 executes control to open the side window 17. By opening the side window 17 in the direction in which the person U1 is located in this way, appropriate notifications can be issued to both the person U21 sitting in the interior space 5 of the vehicle 1 and the person U1.

[0085] Furthermore, notification may be provided by a notification method other than audio output or a side window. For example, assume that the electronic device ED3 carried by the person U1 is a device for which registration information is stored in the storage unit 140. In this case, the device control unit 133 can control the electronic device ED3 via the communication unit 120 to output audio messages similar to the messages M1 and M2 from the electronic device ED3. Furthermore, for example, the device control unit 133 can display information indicating content similar to the messages M1 and M2 on the display unit of the electronic device ED3, along with audio output from the electronic device ED3.

[0086] Also, instead of opening and closing the side windows 15 to 18, the corresponding doors 11 to 14 may be opened or closed or the doors 11 to 14 may be unlocked. If the corresponding door is a sliding door, the sliding door may be opened, and if the corresponding door is not a sliding door, the door may be unlocked. In this way, the person U1 can be notified by opening the door or making a sound to unlock the door.

[0087] Furthermore, if curtains are installed inside the vehicle 1, the curtains closest to the person U1 may be opened.

[0088] Furthermore, the person U1 may be notified using light-emitting units (e.g., turn signals, brake lights, headlights) provided on the exterior of the vehicle 1. For example, the person U1 may be notified by lighting or flashing at least one of the turn signals, brake lights, and headlights. In this case, for example, one or more light-emitting units located near the person U1 may be lit or flashed.

[0089] Furthermore, if it is possible to display an image on the front window 3, rear window 4, or side windows 15 to 18, the notification information may be displayed on at least one of the windows. For example, the notification information may be displayed on one or more windows located close to the person U1. Furthermore, if it is possible to display an image on another part (for example, a door or hood), the notification information may be displayed on that part.

[0090] Furthermore, the notification may be made using at least one of the front wiper 21, the rear wiper 22, and the wing mirrors 31 and 32. For example, the notification can be made by operating the front wiper 21, the rear wiper 22, and the wing mirrors 31 and 32. For example, the notification can be made by operating one or more wipers or wing mirrors located close to the person U1.

[0091] These notification methods are examples of how to provide a predetermined notification to a person U1 outside the vehicle 1, and the predetermined notification may be provided by other notification methods. Furthermore, the notification methods described above may be combined as appropriate, taking into consideration the preferences of the person outside the vehicle 1 and the effect of the notification on that person.

[0092] [Example of notification processing] 9 is a flowchart showing an example of the notification process in the monitoring device 100. The notification process is executed by the control unit 130 based on a program stored in the storage unit 140.

[0093] Furthermore, this notification process is executed at a predetermined timing. This predetermined timing can be, for example, the timing when a person inside the vehicle 1 has no intention of moving the vehicle 1. That is, the vehicle 1 has a state in which the accessory power is on (in other words, a state in which a portion of the power is on) between a state in which the vehicle is ready to run and a state in which the power is completely off. This accessory power on state is, for example, a state in which the vehicle's power is on but the shift lever is in the P (Parking) position. For example, when a person is using the interior space of the vehicle 1 while it is parked, the accessory power is often in the on state. In this accessory power on state, if the vehicle has an internal combustion engine, the engine does not need to be running to run the vehicle, and if the vehicle is an electric vehicle, the high-voltage power to the inverter and drive motor does not need to be on.

[0094] Therefore, in this embodiment, when the power supply of the vehicle 1 is not completely turned off but the accessory power supply is turned on, at least the monitoring device 100 is turned on.

[0095] The predetermined timing may be, for example, the timing when a person U21 inside the vehicle 1 performs an operation to instruct parking (for example, an operation to put the vehicle in P position), the timing when a person U21 inside the vehicle 1 uses a key to unlock the door of the vehicle 1 and then turns on the power of the vehicle 1, the timing when a person U21 inside the vehicle 1 whose power is turned off uses a key to turn on the power of the vehicle 1, etc. Note that Fig. 9 will be explained with reference to the examples shown in Figs. 1 to 8 as appropriate.

[0096] In step S201, the control unit 130 determines whether the vehicle 1 is parked. If the vehicle 1 is parked, the process proceeds to step S202. On the other hand, if the vehicle 1 is not parked, the notification process ends.

[0097] In step S202, the detection unit 131 performs monitoring to detect a moving object around the vehicle 1.

[0098] In step S203, the detection unit 131 determines whether or not a moving object has been detected around the vehicle 1. For example, as shown in FIGS. 4 and 5, if a person U10 or a person U11 is present around the vehicle 1, the person U10 or the person U11 is detected as a moving object present around the vehicle 1. If a moving object is detected around the vehicle 1, the process proceeds to step S204. On the other hand, if a moving object is not detected around the vehicle 1, the process returns to step S202 and continues monitoring.

[0099] In step S204, the determination unit 132 determines whether the moving object detected by the detection unit 131 is moving toward the vehicle 1. For example, as shown in FIG. 4, if a person U10 is moving toward the center of the vehicle 1, it is determined that the moving object (person U10) present around the vehicle 1 is moving toward the vehicle 1. On the other hand, as shown in FIG. 5, if a person U11 is moving around the vehicle 1 but not moving toward the center of the vehicle 1, it is determined that the moving object (person U11) present around the vehicle 1 is not moving toward the vehicle 1. If the moving object is moving toward the vehicle 1, the process proceeds to step S205. On the other hand, if the moving object is not moving toward the vehicle 1, the process proceeds to step S211.

[0100] In step S205, the determination unit 132 determines whether the moving object moving toward the vehicle 1 is carrying a device (registered device) stored as registration information in the storage unit 140. If the moving object is carrying a registered device, the process proceeds to step S208. On the other hand, if the moving object is not carrying a registered device, the process proceeds to step S206.

[0101] In step S206, the determination unit 132 acquires sounds around the vehicle 1 detected by the sound acquisition unit 112. Then, the determination unit 132 performs sound analysis processing on the acquired sounds around the vehicle 1, and determines whether or not the sounds include sounds (registered sounds) stored in the storage unit 140 as registration information. In this case, the direction and position from which the sound is emitted may be determined by analysis. This makes it possible to estimate the direction and position of a moving object. Here, the registered sounds are, as described above, registration information related to sounds such as voiceprints and voice pattern recognition.

[0102] In step S207, the determination unit 132 determines whether a registered sound has been detected from the sounds around the vehicle 1 acquired in step S206. If a registered sound has been detected, the process proceeds to step S208. On the other hand, if a registered sound has not been detected, the process proceeds to step S210. For example, by registering the voiceprints of family members and associates of the person U21 inside the vehicle 1 as registered information, each individual can be identified. Furthermore, for people other than the family members and associates of the person U21, it is possible to determine whether the person U21 has business with them based on the words and sentences uttered by the person.

[0103] The processes of steps S206 and S207 are processes for determining a moving object (person) that does not possess a registered device. For example, since family members and associates of person U21 inside vehicle 1 are assumed to possess registered devices, this process is effective for determining people other than these. However, if someone near vehicle 1 comes to call person U21, it is also assumed that they may approach vehicle 1 without possessing a registered device. For example, a daughter who comes to call her father, person U21, for dinner may not possess a registered device and may not use a specific word, but may simply say the words "Dinner is ready." Therefore, it is preferable to register words frequently used by family members and associates of person U21 (for example, "Dinner is ready," "Documents are ready") as registration information.

[0104] In step S208, the device control unit 133 executes a notification process to notify the person U21 inside the vehicle 1 that a related person has been detected around the vehicle 1. For example, the notification process to notify the person U21 is executed by turning on the light-emitting unit 7 as shown in FIG. 6 or by opening the side window 17 as shown in FIG. 8. In addition, for example, the notification method may be changed based on the content registered as registration information. For example, the notification method when a family member of the person U21 is determined based on the registration information may be different from the notification method when a related person other than the family member of the person U21 is determined based on the registration information, so that the person U21 can easily recognize the people around the vehicle 1.

[0105] In step S209, the device control unit 133 executes a notification process to notify the surroundings of the vehicle 1 that a person inside the vehicle 1 has been notified of the detection of a related person. For example, as shown in Fig. 7, the notification process may be executed to notify the person U1 by outputting a predetermined message M1 by voice, or by opening the side window 17 as shown in Fig. 8.

[0106] In step S210, the control unit 130 determines whether a start operation of the vehicle 1 or a power-off operation to completely turn off the power of the vehicle 1 has been performed. Specifically, the control unit 130 determines whether a person U21 has performed an operation to instruct the parked vehicle 1 to start (for example, an operation to change the position from P to D) and whether a person U21 inside the vehicle 1 has performed a power-off operation to completely turn off the power of the vehicle 1 using a key. If neither a start operation nor a power-off operation of the vehicle 1 has been performed, the process returns to step S202. On the other hand, if a start operation or a power-off operation of the vehicle 1 has been performed, the notification processing operation is terminated.

[0107] In step S211, the determination unit 132 starts counting the detection time.

[0108] In step S212, the determination unit 132 determines whether the detection time is equal to or longer than a predetermined time T1. If the detection time is equal to or longer than the predetermined time T1, the process proceeds to step S216. On the other hand, if the detection time is shorter than the predetermined time T1, the process proceeds to step S213.

[0109] In step S213, the determination unit 132 determines whether the moving object detected in step S203 continues to be detected based on the detection information from the detection unit 131. If the moving object is still being detected, the process proceeds to step S215. On the other hand, if the moving object is no longer detected, the process proceeds to step S214.

[0110] In step S214, the determination unit 132 stops counting the detection time, and the process proceeds to step S210.

[0111] In step S215, the determination unit 132 continues to count the detection time, and the process proceeds to step S212.

[0112] Note that steps S216 to S218 correspond to steps S205 to S207, and therefore a description thereof will be omitted here.

[0113] In step S219, the device control unit 133 executes a notification process to notify a person U21 inside the vehicle 1 that a related person has been detected around the vehicle 1. This notification process may be the same as the notification process of step S208, or may be a different notification process. For example, all light-emitting units present inside the vehicle 1 may be made to emit light, all may be made to flash, or all may be made to a specific color (e.g., yellow or red).

[0114] [Example of notification processing] FIG. 9 shows an example of detecting a moving object using a sensor such as a sonar. Here, because a sensor such as a sonar consumes low power, the power consumption of the vehicle 1 can be effectively utilized. However, when detecting a moving object using images acquired by the image acquisition unit 113, the power consumption used for the image capture process and image analysis process in the image acquisition unit 113 is relatively large. Therefore, it is important to reduce the power consumption while taking advantage of the advantage of detecting a moving object using images acquired by the image acquisition unit 113. Therefore, FIG. 10 shows an example of using images acquired by the image acquisition unit 113 to determine a moving object.

[0115] Fig. 10 is a flowchart showing an example of notification processing in the monitoring device 100. This notification processing is a modified example of the notification processing shown in Fig. 9, and therefore the same process steps as in Fig. 9 are assigned the same reference numerals and their description will be omitted. Note that the power supply to the image acquisition unit 113 that performs imaging processing and image analysis processing may be kept on all the time, or may be turned on when a moving object is detected using a sensor such as sonar, and may be turned off after the determination processing for the moving object is completed.

[0116] In step S202, the detection unit 131 monitors the surroundings of the vehicle 1. In this case, if the power of the image acquisition unit 113 is always on, the object detection process may be performed using the images acquired by the image acquisition unit 113. On the other hand, if the power of the image acquisition unit 113 is not always on, the object detection process is performed using a sensor such as a sonar.

[0117] In step S231, the determination unit 132 determines whether or not a moving object included in the image acquired by the image acquisition unit 113 is registered. In this determination process, a personal identification technique or an individual identification technique is used to determine whether or not a moving object (e.g., a human face) included in the image matches registered information. If the moving object detected around the vehicle 1 is registered, the process proceeds to step S208. On the other hand, if the moving object detected around the vehicle 1 is not registered, the process proceeds to step S211.

[0118] The moving object determination process can be performed quickly by performing the determination process using the image acquired by the image acquisition unit 113. Furthermore, the moving object determination process can be performed using highly accurate personal identification technology or individual identification technology, thereby improving the accuracy of moving object determination.

[0119] As described above, in this embodiment, by executing the notification process by the monitoring device 100, the time it takes for the person U21 inside the vehicle 1 to become aware of the presence of the person U1 outside the vehicle can be significantly reduced. Furthermore, in the notification process, the position of the person U1 outside the vehicle relative to the vehicle 1 (for example, the distance from the vehicle 1 or the direction relative to the vehicle 1) is notified to the person U21 inside the vehicle 1. Therefore, the person U21 inside the vehicle 1 can easily grasp the direction and distance of the person U1 outside the vehicle.

[0120] For example, if the direction and distance of the person U1 outside the vehicle are unknown, the person U21 needs to get outside the vehicle 1, understand the surroundings of the vehicle 1, and then understand the direction and distance of the person U1 outside the vehicle. Furthermore, if the person U21 does not get outside the vehicle 1, it is possible that the person U21 may open a door or side window in a direction in which the person U1 is not present while inside the vehicle 1. In this case, the person U21 needs to repeatedly open and close the door or side window in the direction in which the person U1 is present, which increases the time it takes to encounter the person U1 outside the vehicle. In contrast, if the direction and distance of the person U1 outside the vehicle are known through the notification process described above, the person U21 can quickly open the door or side window in the correct direction in which the person U1 is present. Therefore, the time it takes the person U21 inside the vehicle 1 to search for the direction of the person U1 after noticing the presence of the person U1 outside the vehicle can be significantly reduced, thereby significantly reducing the time it takes to encounter the person U1 outside the vehicle.

[0121] In this way, when a person U1 outside the vehicle approaches the vehicle 1 while immersed in a hobby or work in a situation where they are blocked from the line of sight and sound of the surroundings while parked inside the vehicle 1, the system determines the person U1 who wants to communicate within a predetermined range of the vehicle 1. The system then notifies the person U21 inside the vehicle and the person U1 outside the vehicle of the direction and position of the person U1 through notification processing using a predetermined device, thereby realizing smooth communication.

[0122] [Example of a flying object as a moving object] The above description has been given using an example of a moving object moving on the ground as a moving object existing around vehicle 1. In recent years, transportation using aerial vehicles such as drones has become widespread. As such aerial vehicles, an aerial vehicle that flies using wireless communication or an aerial vehicle that flies autonomously using artificial intelligence (AI) is conceivable. It is therefore conceivable that a person inside a parked vehicle may request the delivery of some kind of luggage, and that the luggage may be delivered by the aerial vehicle. It is also conceivable that a family member may use the aerial vehicle to deliver luggage to a person inside a vehicle parked in a family garden. Therefore, the following description will be given using an aerial vehicle moving in the sky as an example of a moving object existing around a vehicle.

[0123] [Example of the relationship between a vehicle and surrounding flying objects] 11 is a simplified diagram showing the relationship between a parked vehicle 1A and an air vehicle (moving object) 400 existing around the vehicle 1A. In FIG. 11, an example will be described in which the air vehicle 400 moving in the sky is the moving object.

[0124] As shown by an arrow AR31, the flying object 400 is assumed to be a moving object approaching the vehicle 1A in order to deliver luggage to a person U21 (see FIG. 6) inside the vehicle 1A.

[0125] Vehicle 1A is a modified example of vehicle 1, and differs from vehicle 1 in that an openable sunroof 300 is provided on roof 2. Therefore, the following description will focus on the differences from vehicle 1, with the same reference numerals used for the parts common to vehicle 1. Note that sunroof 300 is opened and closed under the control of device control unit 133 (see FIG. 3).

[0126] In addition, moving objects (e.g., flying objects) moving in the air above vehicle 1A can be detected by detecting the moving object using a sensor that uses sound waves, detecting the moving object using sounds in the air above vehicle 1A acquired by sound acquisition unit 112, detecting the moving object using images of the air above vehicle 1A acquired by image acquisition unit 113, detecting the moving object using radio waves emitted by the moving object, etc.

[0127] Furthermore, when determining whether there is a match with registration information for a moving object (e.g., an aircraft) moving in the air above vehicle 1A, this can be performed by making a determination using the sound emitted by the moving object acquired by sound acquisition unit 112, making a determination using an image of the moving object acquired by image acquisition unit 113, or making a determination based on whether the moving object itself is registered using radio waves emitted by the moving object.

[0128] [Example of notification for flying objects around the vehicle] FIG. 12 is a simplified diagram showing the vehicle 1A and the flying object 400 detected around the vehicle 1A.

[0129] As described above, in the example shown in Fig. 12, a predetermined message M11 is output as audio from the vehicle exterior sound output unit 153, and the sunroof 300 on the roof 2 of the vehicle 1A is opened so that the person U21 in the vehicle 1A can receive the luggage carried by the flying object 400. In the case of a vehicle that is not provided with a sunroof 300, the light-emitting state of a light-emitting unit installed on the ceiling of the interior space of the vehicle (for example, the position of the dotted circle 47 shown in Fig. 2) can be controlled to notify the person U21 that the flying object 400 is approaching in the sky. Alternatively, the person U21 may be notified that the flying object 400 is approaching in the sky using the other devices described above (for example, electronic devices, display panels).

[0130] [Configuration and Effects of This Embodiment] The monitoring method according to this embodiment is a monitoring method for monitoring the surroundings of a vehicle 1 (including vehicle 1A; the same applies below). This monitoring method includes a detection step (steps S202 and S203) for detecting a moving object present around the vehicle 1 based on surrounding information around the vehicle 1 (for example, detection values ​​detected by the sensors 111, sounds acquired by the sound acquisition unit 112, images acquired by the image acquisition unit 113, and radio waves acquired via the communication unit 120), and a step of, when a moving object is detected while the vehicle 1 is parked, acquiring information indicating the position of the moving object relative to the vehicle 1 (for example, the direction or orientation of the moving object relative to the vehicle 1, and the distance from the vehicle 1 to the moving object). and control steps (steps S208, S219) for causing predetermined devices (e.g., the front window 3, the rear window 4, the doors 11 to 14, the side windows 15 to 18, the front wiper 21, the rear wiper 22, the wing mirrors 31, 32, the sunroof 300, the light-emitting unit group 151, the in-vehicle sound output unit 152, the outside-vehicle sound output unit 153, the in-vehicle display unit 154, the outside-vehicle display unit 155, the device driving unit group 156, the electronic devices ED1 to ED3, and the vibration device) to execute notification processing for notifying a person U21 inside the vehicle 1 of the above.

[0131] This configuration significantly reduces the time it takes for a person U21 inside the vehicle 1 to become aware of the presence of a moving object (e.g., a person) outside the vehicle, thereby enabling smooth communication from outside with the person U21 inside the vehicle 1.

[0132] Furthermore, in the monitoring method according to this embodiment, when a moving object is detected in the detection step (steps S202, S203) while the vehicle 1 is parked, the method further includes a determination step (steps S204 to S207, S211 to S218) for determining whether the moving object is related to a person U21 inside the vehicle 1 based on at least one of the characteristics of the moving object's behavior and the degree of coincidence between information about the moving object and the registration information. Furthermore, in the control step (steps S208, S219), if the moving object detected in the detection step (steps S202, S203) is determined to be related to a person U21 inside the vehicle 1 in the determination step, a notification process is executed by a predetermined device.

[0133] According to this configuration, notification processing for person U21 is executed only when a moving object related to person U21 inside vehicle 1 is detected, thereby preventing notification processing from being executed every time a moving object that simply passes by vehicle 1 is detected.

[0134] In addition, in the monitoring method of this embodiment, the judgment step (steps S204, S211 to S215) determines whether the behavior of the moving object detected in the detection step (steps S202, S203) corresponds to a first pattern in which the moving object moves toward the center of vehicle 1, or a second pattern in which the moving object stays within a predetermined range R1 based on vehicle 1 for a predetermined time T1 or more, and if the behavior of the moving object corresponds to the first pattern or the second pattern, it is determined that the moving object is related to a person U21 inside vehicle 1.

[0135] According to this configuration, since the determination process can be performed using appropriate determination conditions as characteristics of the behavior of the moving object, the accuracy of determining the moving object related to the person U21 inside the vehicle 1 can be improved.

[0136] Furthermore, in the monitoring method of this embodiment, in the judgment step (steps S204 to S207, S211 to S218), if the behavior of the moving object detected in the detection step (steps S202, S203) corresponds to the first pattern or the second pattern, the degree of match between the information about the moving object and the registered information is further determined as whether the moving object matches the registered information (for example, whether the person's face matches the registered face), whether the device carried by the moving object matches the registered information (for example, whether the smartphone matches the registered device), or whether the sound made by the moving object matches the registered information (for example, whether the person's voiceprint matches the registered sound), and if at least one of these matches, it is determined that the moving object is related to person U21 inside vehicle 1.

[0137] According to this configuration, the judgment process can be performed based on the behavioral characteristics of the moving object, as well as using appropriate judgment conditions based on the degree of agreement between information about the moving object and registered information, thereby further improving the accuracy of judgment of moving objects related to person U21 inside vehicle 1.

[0138] In addition, in the monitoring method of this embodiment, in the judgment steps (steps S205 to S207, S216 to S218), the degree of match between the information about the moving object detected in the detection steps (steps S202, S203) and the registration information is determined by determining whether the moving object matches the registration information, whether the equipment carried by the moving object matches the registration information, or whether the sound emitted by the moving object matches the registration information, and if it is determined that at least one of these matches, it is determined that the moving object is related to a person U21 inside vehicle 1.

[0139] According to this configuration, the determination process can be performed using registration information registered in advance by the person U21, and therefore the accuracy of determining a moving object related to the person U21 inside the vehicle 1 can be improved.

[0140] Furthermore, in the monitoring method of this embodiment, in the control steps (steps S208, S219), the notification method for the notification process when it is determined that the moving object detected in the detection steps (steps S202, S203) is moving toward the center of vehicle 1 as a behavioral characteristic is different from the notification method for the notification process when it is determined that the moving object has remained within a predetermined range R1 based on vehicle 1 for a predetermined time T1 or more.

[0141] According to this configuration, the person U21 inside the vehicle 1 can be notified using different notification methods depending on the characteristics of the moving object's behavior, so that the person U21 can quickly begin taking action against the moving object outside in response to the notification.

[0142] Furthermore, in the monitoring method according to this embodiment, the predetermined devices are a plurality of light-emitting units (positions indicated by dotted circles 41 to 46 in FIG. 2 (including light-emitting units 6 to 8 in FIG. 6)) installed inside the vehicle 1 in a plurality of directions including the front, rear, left, and right of the vehicle 1. Furthermore, in the control step (steps S208 and S219), if a moving object is detected in the detection step (steps S202 and S203), a notification process is executed by changing the light-emitting state of the light-emitting unit corresponding to the direction specified by the position of the moving object relative to the vehicle 1 to a different state from the light-emitting state of the other light-emitting units. For example, if a person U1 is detected as shown in FIG. 1, only light-emitting unit 7 is made to emit light as shown in FIG. 6, and a notification process is executed. In addition, instead of the light-emitting unit, a notification process may be performed using multiple vibration devices arranged in the interior space of the vehicle 1, the front windshield 3, the rear windshield 4, the doors 11 to 14, the side windows 15 to 18, the front wiper 21, the rear wiper 22, the wing mirrors 31, 32, the sunroof 300, etc., to notify the direction identified by the position of a moving object relative to the vehicle 1.

[0143] This configuration allows a person U21 inside the vehicle 1 to easily visually grasp the position of the person U1 outside the vehicle relative to the vehicle 1 (for example, the distance from the vehicle 1 or the direction relative to the vehicle 1). This allows the person U21 to quickly open the door or side window in the correct direction where the person U1 is located, significantly reducing the time it takes to encounter the person U1 outside the vehicle.

[0144] Moreover, in the monitoring method according to this embodiment, the predetermined device is the interior sound output unit 152 (an example of a sound output unit) installed inside the vehicle 1. Furthermore, in the control step (steps S208, S219), if a moving object is detected in the detection step (steps S202, S203), a notification process is executed by causing the interior sound output unit 152 to output information indicating the position of the moving object relative to the vehicle 1 as a voice. For example, as shown in FIG. 1, if a person U1 is detected, the interior sound output unit 152 can output a voice message such as "An interested person is approaching from the rear left."

[0145] This configuration allows a person U21 inside the vehicle 1 to easily and auditorily grasp the position of the person U1 outside the vehicle relative to the vehicle 1 (for example, the distance from the vehicle 1 or the direction relative to the vehicle 1). This allows the person U21 to quickly open the door or side window in the correct direction where the person U1 is located, significantly reducing the time it takes to encounter the person U1 outside the vehicle.

[0146] Moreover, in the monitoring method according to this embodiment, the predetermined devices are the electronic devices ED1 and ED2 used inside the vehicle 1. Furthermore, in the control step (steps S208 and S219), if a moving object is detected in the detection step (steps S202 and S203), a notification process is performed by causing the electronic devices ED1 and ED2 to output audio or display an image of information indicating the position of the moving object relative to the vehicle 1. For example, as shown in FIG. 6, if the electronic devices ED1 and ED2 are being used by a person U21 inside the vehicle 1, the electronic device ED1 can output audio such as a message "Someone is approaching from the rear left," and the display unit of the electronic device ED2 can display a message such as "Someone is approaching from the rear left."

[0147] According to this configuration, the position of person U1 outside the vehicle relative to vehicle 1 (for example, the distance from vehicle 1 or the direction relative to vehicle 1) can be easily determined from electronic devices ED1 and ED2 used by person U21 inside vehicle 1. This allows the door or side window in the correct direction where person U1 is located to be quickly opened, significantly reducing the time it takes to encounter person U1 outside the vehicle.

[0148] In addition, in the monitoring method of this embodiment, in the control step (step S209), if a moving object is detected in the detection step (steps S202, S203), notification processing is performed in steps S208 and S219, and external notification processing is also performed to notify the outside of vehicle 1 of external notification information indicating that the detection of the moving object has been notified to a person U21 inside vehicle 1.

[0149] With this configuration, person U1 outside vehicle 1 can easily understand that person U21 inside vehicle 1 is being notified of person U1's presence, and can therefore easily understand that person U21 will come out of vehicle 1 if he waits a little while.

[0150] Furthermore, in the monitoring method according to this embodiment, the vehicle 1 is provided with an exterior sound output unit 153 that outputs sound to the outside of the vehicle 1. Furthermore, in the control step (step S209), if a moving object is detected in the detection steps (steps S202 and S203), an exterior notification process is executed by causing the exterior sound output unit 153 to output exterior notification information as sound. For example, as shown in FIGS. 7 and 8, the exterior notification process can be executed by outputting predetermined messages M1 and M2 as sound from the exterior sound output unit 153.

[0151] According to this configuration, the person U1 outside the vehicle 1 can easily audibly understand that the presence of the person U1 is being notified to the person U21 inside the vehicle 1.

[0152] Furthermore, in the monitoring method according to this embodiment, the vehicle 1 includes a communication unit 120 that exchanges information via wireless communication with an electronic device ED3 (an example of a device) used outside the vehicle 1. Furthermore, in a control step (step S209), if a moving object is detected in the detection steps (steps S202 and S203), an external notification process is executed by causing the electronic device ED3 to output audio or display an image of external notification information. For example, as shown in FIGS. 7 and 8, if a person U1 approaching the vehicle 1 is carrying the electronic device ED3, the external notification process can be executed by audio outputting predetermined messages M1 and M2 from the audio output unit of the electronic device ED3 or by displaying a predetermined message on the display unit of the electronic device ED3.

[0153] According to this configuration, a person U1 outside the vehicle 1 can easily understand that the presence of the person U1 is being notified to a person U21 inside the vehicle 1 through an external notification process using the electronic device ED3 that the person U1 possesses.

[0154] The monitoring device 100 is a monitoring device that monitors the surroundings of the vehicle 1. The monitoring device 100 also includes a detection unit 131 that detects a moving object present around the vehicle 1 based on surrounding information about the surroundings of the vehicle 1, and an equipment control unit that causes a predetermined device to execute a notification process for notifying a person U21 inside the vehicle 1 of information indicating the position of the moving object relative to the vehicle 1 when the moving object is detected while the vehicle 1 is parked.

[0155] This configuration significantly reduces the time it takes for a person U21 inside the vehicle 1 to become aware of the presence of a moving object (e.g., a person) outside the vehicle, thereby enabling smooth communication from outside with the person U21 inside the vehicle 1.

[0156] In the present embodiment, the monitoring device 100 performs the moving object detection process and the moving object determination process. However, all or part of these processes may be performed by other devices. In this case, the monitoring system for the vehicle 1 includes devices that perform part of these processes. For example, at least part of the moving object detection process and the moving object determination process may be performed using an electronic device ED2 (see FIG. 6 ) used by a person U21 inside the vehicle 1, various information processing devices such as servers connectable via a predetermined network such as the Internet, and various electronic devices. When the electronic device ED2 is used, the control unit 130 transmits information acquired by at least one of the sensors 111, the sound acquisition unit 112, and the image acquisition unit 113 to the electronic device ED2 via the communication unit 120. Then, the control unit of the electronic device ED2 performs the moving object detection process and the moving object determination process described in this embodiment based on the information, and transmits the detection results and determination results to the control unit 130 via the communication unit 120. Next, the device control unit 133 causes each device to perform a notification process using the detection results and determination results transmitted from the electronic device E2. If the electronic device ED2 is capable of executing a moving object detection process and a moving object determination process based on radio waves received from another device, the control unit of the electronic device ED2 may execute the moving object detection process and the moving object determination process described in this embodiment based on the radio waves, and transmit the detection results and determination results to the control unit 130 via the communication unit 120. Furthermore, the display unit and sound output unit of the electronic device ED2 may execute a notification process based on the detection results and determination results. In these cases, the electronic device ED2 functions as a monitoring device for the vehicle 1.

[0157] Furthermore, in this embodiment, an example is shown in which registration information used when executing the moving object determination process is stored and managed in the storage unit 140 provided in the monitoring device 100. However, all or part of the registration information may be managed by one or more devices other than the monitoring device 100, and the monitoring device 100 may acquire and use the registration information managed by the other devices. For example, the registration information may be stored and managed in a storage unit of the electronic device ED2, or may be stored and managed in a storage unit of an information processing device such as a server connectable via a predetermined network such as the Internet.

[0158] Furthermore, a part (or all) of the monitoring system of the vehicle 1 that can execute the functions of the monitoring device 100 may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0159] Each process described in this embodiment is executed based on a program that causes a computer to execute each processing procedure. Therefore, this embodiment can also be understood as an embodiment of a program that realizes the function of executing each process and a recording medium that stores the program. For example, an update process for adding a new function to a monitoring device can store the program in the storage device of the monitoring device. This makes it possible to cause the updated monitoring device to execute each process described in this embodiment.

[0160] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments. [Explanation of symbols]

[0161] 1 vehicle, 2 roof, 3 front window, 4 rear window, 5 interior space, 6-8 light emitting unit, 11-14 door, 15-18 side window, 21 front wiper, 22 rear wiper, 31, 32 wing mirror, 51-60 ultrasonic sensor, 100 monitoring device, 111 sensors, 112 sound acquisition unit, 113 image acquisition unit, 120 communication unit, 130 control unit, 131 detection unit, 132 determination unit, 133 equipment control unit, 140 memory unit, 151 light emitting unit group, 152 interior sound output unit, 153 exterior sound output unit, 154 interior display unit, 155 exterior display unit, 156 equipment drive unit group, 300 sunroof, 400 flying object

Claims

1. A monitoring method for monitoring an area around a vehicle by a computer, comprising: a detection step of detecting a moving object existing around the vehicle based on surrounding information about the vehicle; a control step of causing a predetermined device to execute a notification process for notifying a person inside the vehicle of information indicating the position of the moving object relative to the vehicle when the moving object is detected while the vehicle is parked; In the control step, a notification method of the notification process when it is determined that the detected moving object is moving toward the center of the vehicle is different from a notification method of the notification process when it is determined that the moving object has stayed within a predetermined range based on the vehicle for a predetermined period of time or more. Monitoring method.

2. A monitoring method for monitoring, by a computer, the surroundings of a vehicle having a communication unit that exchanges information with an external device used outside the vehicle by wireless communication, comprising: a detection step of detecting a moving object existing around the vehicle based on surrounding information about the vehicle; a control step of causing a predetermined device to execute a notification process for notifying a person inside the vehicle of information indicating the position of the moving object relative to the vehicle when the moving object is detected while the vehicle is parked; In the control step, an external notification process is executed to notify the outside of the vehicle by causing the external device to output a voice or display an image of external notification information indicating that the detection of the moving object has been notified to a person inside the vehicle. Monitoring method.

3. 3. The monitoring method according to claim 1 or 2, If a moving object is detected in the detection step while the vehicle is parked, the computer is further caused to execute a determination step of determining whether the moving object is related to a person inside the vehicle based on at least one of the characteristics of the behavior of the moving object and the degree of coincidence between the information about the moving object and the registered information; In the control step, when it is determined in the determination step that the moving object detected in the detection step is related to a person inside the vehicle, the predetermined device is caused to execute the notification process. Monitoring method.

4. 4. The monitoring method according to claim 3, In the determination step, the characteristics of the behavior of the moving object detected in the detection step are determined as to whether the moving object corresponds to a first pattern in which the moving object moves toward the center of the vehicle, or whether the moving object corresponds to a second pattern in which the moving object stays within a predetermined range based on the vehicle for a predetermined period of time or more, and if the behavior of the moving object corresponds to the first pattern or the second pattern, it is determined that the moving object is related to a person inside the vehicle. Monitoring method.

5. 5. The monitoring method according to claim 4, In the determination step, if the behavior of the moving object detected in the detection step corresponds to the first pattern or the second pattern, the degree of match between the information about the moving object and the registration information is further determined by determining whether the moving object matches the registration information, whether the equipment carried by the moving object matches the registration information, or whether the sound emitted by the moving object matches the registration information, and if at least one of these matches, it is determined that the moving object is related to a person inside the vehicle. Monitoring method.

6. 4. The monitoring method according to claim 3, In the determination step, the degree of match between the information about the moving object detected in the detection step and the registration information is determined by determining whether the moving object matches the registration information, whether the equipment carried by the moving object matches the registration information, or whether the sound emitted by the moving object matches the registration information, and if it is determined that at least one of these matches, it is determined that the moving object is related to a person inside the vehicle. Monitoring method.

7. 7. A monitoring method according to any one of claims 1 to 6, comprising: the predetermined devices are a plurality of light-emitting units installed inside the vehicle in a plurality of directions including the front, rear, left, and right of the vehicle, In the control step, when a moving object is detected in the detection step, the notification process is executed by changing the light emission state of the light emitting unit corresponding to a direction specified by the position of the moving object relative to the vehicle to a state different from the light emission states of other light emitting units. Monitoring method.

8. 8. A monitoring method according to any one of claims 1 to 7, comprising: the predetermined device is a sound output unit installed inside the vehicle, In the control step, when a moving object is detected in the detection step, the notification process is executed by causing the sound output unit to output, by voice, information indicating the position of the moving object relative to the vehicle. Monitoring method.

9. 9. A monitoring method according to any one of claims 1 to 8, comprising: the predetermined device is an electronic device used inside the vehicle, In the control step, when a moving object is detected in the detection step, the notification process is performed by causing the electronic device to output a voice or display an image of information indicating the position of the moving object relative to the vehicle. Monitoring method.

10. 2. The monitoring method of claim 1, In the control step, when a moving object is detected in the detection step, the notification process is executed, and an external notification process is executed to notify an outside of the vehicle of external notification information indicating that the detection of the moving object has been notified to a person inside the vehicle. Monitoring method.

11. 11. The monitoring method of claim 10, the vehicle includes an exterior sound output unit that outputs sound to the outside of the vehicle; In the control step, when a moving object is detected in the detection step, the external notification process is executed by outputting the external notification information as sound from the vehicle external sound output unit. Monitoring method.

12. A monitoring device for monitoring the surroundings of a vehicle, a detection unit that detects a moving object existing around the vehicle based on surrounding information about the vehicle; and a device control unit that, when the moving object is detected while the vehicle is parked, causes a predetermined device to execute a notification process for notifying a person inside the vehicle of information indicating the position of the moving object relative to the vehicle, the device control unit uses different notification methods for the notification process when it is determined that the detected moving object is moving toward the center of the vehicle and when it is determined that the moving object has stayed within a predetermined range based on the vehicle for a predetermined period of time or more. monitoring equipment.

13. A monitoring device for monitoring the surroundings of a vehicle, a communication unit that exchanges information with an external device used outside the vehicle by wireless communication; a detection unit that detects a moving object existing around the vehicle based on surrounding information about the vehicle; and a device control unit that, when the moving object is detected while the vehicle is parked, causes a predetermined device to execute a notification process for notifying a person inside the vehicle of information indicating the position of the moving object relative to the vehicle, the device control unit executes an external notification process for notifying the outside of the vehicle by causing the external device to output a sound or display an image of external notification information indicating that the detection of the moving object has been notified to a person inside the vehicle. monitoring equipment.

Images