Abandonment determination system, abandonment determination device, and abandonment determination method

Abstract

The present invention comprises: a biological information acquisition unit (1301) for acquiring biological information; a three-dimensional image acquisition unit (1302) for acquiring a three-dimensional image; a vehicle interior determination unit (131) for performing vehicle interior determination, which is to determine whether a moving body considered to be a person in need of assistance is present in the vehicle interior, on the basis of the three-dimensional image and the biological information; a vehicle exterior determination unit (135) for performing vehicle exterior determination, which is to determine whether a moving body considered to be a person is present outside the vehicle exterior, on the basis of the three-dimensional image; and an abandonment determination unit (14) for determining whether an abandonment of a person in need of assistance has occurred in the vehicle interior, on the basis of the determination result of the vehicle interior determination made by the vehicle interior determination unit (131) and the determination result of the vehicle exterior determination made by the vehicle exterior determination unit (135).

Description

[Technical Field]

[0001] The present disclosure relates to an abandonment determination system, an abandonment determination device, and an abandonment determination method. [Background technology]

[0002] Conventionally, a technology has been known in which a radio wave sensor is installed in the vehicle cabin, emits radio waves into the cabin, and receives reflected waves from objects to detect the position and movement of moving objects in the cabin, thereby determining whether or not a child or other person has been left behind in the vehicle cabin based on information generated from the reflected waves received by the radio wave sensor. Incidentally, a technology is known in which a biosensor is used as a sensor for determining whether or not a child or other person has been left behind in the vehicle cabin, and detects the biological signals of a child or other person present in the vehicle cabin from the movement of the body surface, and determines whether or not a child or other person has been left behind in the vehicle cabin based on the biological information from the biosensor (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-149485 Summary of the Invention [Problem to be solved by the invention]

[0004] The radio waves emitted by the radio wave sensor have the property of being transmitted, reflected, or diffracted depending on the materials of the structures and the like inside the vehicle interior, so the radio wave sensor can detect the position and movement of a moving object inside the vehicle interior over a wide range within the vehicle interior. Therefore, by determining whether or not a child or other person has been left behind inside the vehicle interior based on information generated from the reflected waves received by the radio wave sensor, it becomes possible to determine whether or not a child or other person has been left behind inside the vehicle interior over a wide range within the vehicle interior.

[0005] On the other hand, it is difficult to control the detection range of a radio wave sensor to an appropriate range. Therefore, radio waves emitted by a radio wave sensor installed inside the vehicle cabin may propagate outside the vehicle cabin, even if the radio wave sensor is not intended to do so. In other words, a radio wave sensor installed inside the vehicle cabin may also detect a person or other object outside the vehicle cabin. Therefore, due to the radio wave sensor's properties of transmission, reflection, or diffraction, the radio wave sensor may determine that a person or other object outside the vehicle cabin is actually inside the vehicle cabin.

[0006] In the conventional technology described above, which determines whether or not a child or other person has been left behind in the vehicle cabin based on information generated from the reflected waves received by a radio wave sensor that is installed in the vehicle cabin, emits radio waves into the cabin, and can detect the position and movement of moving objects in the cabin by receiving reflected waves from the emitted radio waves reflected by objects, there was a problem in that the nature of the radio wave sensor described above could result in an incorrect determination of whether or not a child or other person has been left behind in the vehicle cabin. In addition, the technology for determining whether an infant or other child has been left behind using a biosensor, as disclosed in Patent Document 1, does not take into account the possibility of incorrect determination of whether an infant or other child has been left behind due to the properties of the radio wave sensor, as described above, and therefore is still unable to solve the above problem.

[0007] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an abandonment determination system that prevents erroneous determination of whether or not a child or other person has been left behind in a vehicle. [Means for solving the problem]

[0008] The abandoned vehicle determination system according to the present disclosure is installed in a vehicle interior, and the radio waves emitted toward the vehicle interior are detected by the and is located within a range where radio waves can be emitted, including outside the vehicle. a radio wave sensor that detects a moving object based on a reflected wave reflected by an object; a biometric information acquisition unit that acquires biometric information extracted based on the detection result of the moving object detected by the radio wave sensor; and a vehicle interior biometric information acquisition unit that acquires biometric information extracted based on the detection result of the moving object detected by the radio wave sensor. and the range where radio waves can be emitted, including outside the vehicle.a three-dimensional image acquisition unit that acquires a three-dimensional image that shows the distribution of an area where a moving object is present in three dimensions; a vehicle interior determination unit that determines whether or not a moving object that is deemed to be a person requiring assistance is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit and the biometric information acquired by the biometric information acquisition unit; and By determining whether or not there is a moving object that can be considered a person in the area outside the vehicle cabin of the 3D image, The vehicle is provided with an outside-cabin determination unit that performs an outside-cabin determination to determine whether or not there is a moving object that can be regarded as a person outside the vehicle, and an abandonment determination unit that determines whether or not a person requiring assistance has been left behind inside the vehicle based on the results of the inside-cabin determination performed by the inside-cabin determination unit and the results of the outside-cabin determination performed by the outside-cabin determination unit. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to prevent erroneous determination of whether or not a child or the like has been left behind in the vehicle interior. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a configuration example of an abandonment determination system according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of the configuration of a biometric determination device according to a first embodiment. [Figure 3] 1 is a diagram illustrating an example of the configuration of an abandonment determination device according to a first embodiment. [Figure 4] FIG. 2 is a diagram for explaining in detail the radio wave sensor according to the first embodiment. [Figure 5] FIG. 2 is a diagram for explaining in detail the radio wave sensor according to the first embodiment. [Figure 6] 3 is a diagram for explaining the flow of signals in a radio wave transmitting and receiving section of the radio wave sensor according to the first embodiment. FIG. [Figure 7] FIG. 7A is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave, and FIG. 7B is an explanatory diagram showing the relationship between the transmission and reception period of the FM transmission wave and the frequency difference fd. [Figure 8] 8A, 8B, and 8C are diagrams showing examples of three-dimensional images generated by the three-dimensional image generating unit in the first embodiment. [Figure 9] FIG. 10 is a diagram showing Doppler components when a moving object (occupant) is present and when no moving object (occupant) is present. [Figure 10] FIG. 10 is a diagram showing an example of a first abandonment determination table used by a first determination unit in the first abandonment determination process in the first embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a second abandonment determination table used by the second determination unit in the second abandonment determination process in the first embodiment. [Figure 12] 12A and 12B are diagrams showing examples of a first alarm table and a second alarm table used by the alarm control unit to determine whether to output an alarm instruction in embodiment 1, where FIG. 12A is a diagram showing an example of the first alarm table and FIG. 12B is a diagram showing an example of the second alarm table. [Figure 13] 4 is a flowchart for explaining the operation of the abandonment determination system according to the first embodiment. [Figure 14] 14 is a flowchart for explaining in detail the processing of step ST2 in FIG. 13. [Figure 15] 4 is a flowchart for explaining the operation of the alarm control device according to the first embodiment. [Figure 16] 16A and 16B are diagrams illustrating an example of a hardware configuration of the abandonment determination device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0012] In recent years, occupant monitoring systems that use sensors to detect occupants in the vehicle cabin and determine their behavior have been introduced to the market. Typical functions of such occupant monitoring systems include, for example, a function to detect driver behavior such as drowsiness or inattentive driving, primarily for the purpose of driving assistance. Furthermore, technological and product development of occupant monitoring systems that have the function to detect the seating positions or physiques of all occupants in the vehicle cabin is progressing, with the aim of further enhancing vehicle safety. For example, seat belt reminders for all seats or systems that determine whether small children or other infants have been left behind in the vehicle cabin are being developed. In particular, leaving young children behind in vehicles has become a social problem, and Euro NCAP (New Car Assessment Programme), which handles automobile assessments in Europe, plans to add "Child Presence Detection" to its test items in 2022.

[0013] Here, a camera is one example of a sensor that is currently in practical use in a vehicle interior. The images captured by the camera can not only detect the presence or absence of an occupant in the vehicle cabin or determine the physique of the occupant, but also enable advanced detection or determination, such as personal authentication through facial recognition or detection of drowsiness by checking whether the eyes are open or closed. However, as described above, the detection of the presence or absence of an occupant in the vehicle cabin or the determination of the occupant's physical size is performed using captured images within the camera's visible range, so the detection range or determination range is limited.

[0014] Another example of a sensor that is put to practical use in a vehicle interior is a radio wave sensor. The radio wave sensor generates a 3D image by receiving and processing the reflected radio waves that are reflected by the target object (more specifically, the passenger). This 3D image makes it possible to detect the presence or absence of a passenger in the vehicle, determine the passenger's seating position, determine the passenger's physique, or detect the passenger's movement. The radio wave sensor also extracts minute movements, such as the breathing of the occupant, from the reflected waves as biometric information, which enables the presence or absence of an object (more specifically, an occupant) to be determined.

[0015] Note that there is a limit to the range of radio wave irradiation, which is the detection range of the radio wave sensor. However, unlike the visible light or near-infrared light sensed by a camera, the radio waves emitted by a radio wave sensor have the property of transmitting, reflecting, or diffracting depending on the materials of the interior structures of the vehicle, such as seats or the ceiling (hereinafter referred to as "vehicle structures"). Therefore, by widening the antenna directivity of the radio wave sensor and optimizing its installation position, the radio wave sensor has the characteristic of being able to realize a wide detection range. The antenna directivity of the radio wave sensor corresponds to the lens angle of the camera.

[0016] As described above, the radio wave sensor can have a wide detection range. However, on the other hand, it is difficult to control the detection range of a radio wave sensor to an appropriate range. For example, radio waves emitted from a radio wave sensor installed in the vehicle interior to monitor passengers inside the vehicle may propagate outside the vehicle interior, which is not the intended purpose. Therefore, the radio wave sensor detects people around the vehicle, in other words, people outside the vehicle interior. Details of how the radio wave sensor detects people outside the vehicle interior will be described later with reference to the drawings.

[0017] When a radio wave sensor detects a person outside the vehicle cabin, and a determination is made as to whether a child or other person has been left behind in the vehicle cabin using a 3D image or biometric information generated based on the results of the radio wave sensor detecting an object (more specifically, an occupant), it may be determined that a child or other person has been left behind in the vehicle cabin even though they are not.

[0018] The present disclosure provides an abandonment determination system that uses information from a radio wave sensor (more specifically, three-dimensional images and biometric information) to determine whether a person requiring assistance, such as an infant, has been left behind in a vehicle cabin, making it possible to determine whether the person has been left behind over a wide range, and that prevents erroneous determination of whether a person requiring assistance has been left behind due to the properties of the radio wave sensor, as described above. In this disclosure, the term "person requiring assistance" refers to a living being, such as a small child, who would have difficulty getting out of a vehicle on his or her own if left behind in the vehicle. In this disclosure, the term "person requiring assistance" also includes living beings, such as pets. In the following description, as an example, the "person requiring assistance" is an infant, and the abandonment determination system determines whether an infant has been left behind in the vehicle.

[0019] Embodiment 1 FIG. 1 is a diagram showing an example of the configuration of an abandonment determination system 100 according to the first embodiment. The abandonment determination system 100 is composed of a biometric determination device 130 and an abandonment determination device 140. The abandonment determination device 140 is mounted on a vehicle and is connected to the living body determination device 130, the alarm control device 150, and the vehicle information device 40.

[0020] The biometric determination device 130 is mounted on a radio wave sensor 11 provided inside the vehicle. The biometric determination device 130 determines whether or not a moving object deemed to be an infant is present inside the vehicle cabin (hereinafter referred to as "inside vehicle cabin determination") and whether or not a moving object is present outside the vehicle cabin (hereinafter referred to as "outside vehicle cabin determination") based on the biometric information generated based on the detection result of the moving object detected by the radio wave sensor 11 and the 3D image. In the first embodiment, "moving object" refers to a person, including a vehicle occupant. The radio wave sensor 11 will be described in detail later. 2 is a diagram showing an example of the configuration of the biometric determination device 130 according to Embodiment 1. Details of the example of the configuration of the biometric determination device 130 will be described later.

[0021] The vehicle information device 40 is a device that is mounted on a vehicle and is capable of communicating with a short-range wireless communication device 45 . The short-range wireless communication device 45 is, for example, a mobile terminal carried by a person such as a vehicle owner, a vehicle key having a so-called keyless entry function, or a so-called smart key. Here, the vehicle information device 40 plays a role in controlling the locking and unlocking of the vehicle doors by exchanging key information with the vehicle key, for example. The vehicle information device 40 includes a wireless communication status monitoring unit 41. The wireless communication status monitoring unit 41 monitors the communication status with the short-range wireless communication device 45. More specifically, the wireless communication status monitoring unit 41 monitors whether wireless communication between the vehicle and the short-range wireless communication device 45 has been established. The wireless communication status monitoring unit 41 outputs information (hereinafter referred to as "communication status information") relating to the communication status between the vehicle and the short-range wireless communication device 45, i.e., whether wireless communication between the vehicle and the short-range wireless communication device 45 has been established, to the abandoned vehicle determination device 140.

[0022] The abandonment determination device 140 determines whether or not a child has been left behind inside the vehicle based on the results of the vehicle interior determination and the vehicle exterior determination performed by the biometric determination device 130 and the communication status information obtained from the vehicle information device 40. 3 is a diagram showing an example of the configuration of the abandonment determination device 140 according to Embodiment 1. The details of the example of the configuration of the abandonment determination device 140 will be described later.

[0023] The alarm control device 150 is mounted on the vehicle and causes the alarm device 160 to output an alarm based on the determination result by the abandonment determination device 140 as to whether or not a child has been left behind in the vehicle compartment. The warning device 160 is assumed to be, for example, a display device such as a display or an audio output device such as a speaker provided on a mobile terminal carried by the vehicle owner, or a lamp or horn provided on the vehicle. An example of the configuration of the alarm control device 150 will be described in detail later.

[0024] Here, the radio wave sensor 11 will be described in detail. The radio wave sensor 11 according to the first embodiment is equipped with an antenna with wide-angle directivity, and is installed inside the vehicle interior so that radio waves are irradiated to passengers who may be present inside the vehicle interior.

[0025] 4 and 5 are diagrams for explaining the radio wave sensor 11 according to the first embodiment in detail. FIG. 4 is a diagram showing an example of the interior of a vehicle as seen from the side of the vehicle. FIG. 5 is a diagram showing an example of the interior and exterior of the vehicle when viewed from above. In Figures 4 and 5, the vehicle is designated "10". In the three-dimensional coordinate system representing the real space inside the vehicle cabin, the x-axis is an axis parallel to the vehicle width direction, the y-axis is an axis parallel to the vehicle height direction, and the z-axis is an axis parallel to the vehicle length direction. Note that in the first embodiment, "parallel" is not limited to being strictly "parallel" and includes "approximately parallel." 4 and 5 are right-hand drive vehicles, but this is merely an example, and the vehicle may also be a left-hand drive vehicle. As shown in FIGS. 4 and 5, the radio wave sensor 11 is installed, for example, in the upper part of the vehicle interior.

[0026] Figure 4 shows the interior of a vehicle when there is an adult passenger 30 in the passenger seat and a passenger (more specifically, a child) 31 lying in a child seat 20 installed facing backwards in the rear seat. In the vehicle interior shown in FIG. 4, the radio wave sensor 11 emits radio waves toward the occupants 30 and 31 inside the vehicle interior, and receives the radio wave reflected by the occupants 30 and 31. In FIG. 4, the image of the radio wave emitted from the radio wave sensor 11 and the reflected waves of the radio wave reflected by the occupant 30 and the occupant 31 is indicated by "13."

[0027] 5 shows the state inside and outside the vehicle when an adult occupant 35 who was inside the vehicle gets out of the vehicle. There is no occupant inside the vehicle. The occupant 35 is assumed to be, for example, the owner of the vehicle. The occupant 35 carries a short-range wireless communication device 45 and moves between within the communication range of the short-range wireless communication device 45 (indicated by "46" in FIG. 5) and outside the communication range of the short-range wireless communication device 45. For convenience, FIG. 5 illustrates both the occupant 35 who is within the communication range of the short-range wireless communication device 45 and the occupant 35 who is outside the communication range of the short-range wireless communication device 45. As described above, the radio wave sensor 11 is provided inside the vehicle interior so that radio waves are irradiated to passengers who may be present inside the vehicle interior. However, as described above, it is difficult for the radio wave sensor 11 to control the detection range to an appropriate range. Therefore, the radio waves emitted by the radio wave sensor 11 may also be emitted outside the vehicle cabin. In Figure 5, the radiation range of the radio waves emitted by the radio wave sensor 11 is indicated by "15." The radiation range into which the radio wave sensor 11 emits radio waves can also be said to be the detection range within which the radio wave sensor 11 can detect an object.

[0028] When the radio waves emitted by the radio wave sensor 11 are also emitted outside the vehicle, the radio waves are also emitted to a person outside the vehicle. As a result, the radio wave sensor 11 may process the reflected waves of the radio waves emitted to the person outside the vehicle as signals of radio waves reflected by an occupant inside the vehicle.

[0029] Signal processing by the radio wave sensor 11 will now be described. As shown in FIG. 1, the radio wave sensor 11 includes a radio wave transmitting / receiving unit 110, a sensor signal processing unit 120, and a living body determination device . The radio wave transmitting / receiving section 110 is made up of a transmitting antenna 111, a receiving antenna 112, a high frequency signal generating circuit section 113, a transmitting circuit section 114, a receiving circuit section 115, and an A / D conversion section .

[0030] The transmitting antenna 111 is a planar antenna configured on an electronic circuit board. The transmitting antenna 111 has a plurality of transmitting antenna elements (not shown) that radiate radio waves toward a target area.

[0031] The receiving antenna 112 is a planar antenna configured on an electronic circuit board. The receiving antenna 112 has a plurality of receiving antenna elements (not shown) that receive the reflected waves of the radio waves emitted from the transmitting antenna 111. Although only one transmitting antenna 111, one receiving antenna 112, one transmitting circuit unit 114, and one receiving circuit unit 115 are shown in FIG. 1, the transmitting antenna 111, the receiving antenna 112, the transmitting circuit unit 114, or the receiving circuit unit 115 may be configured with multiple units in order to meet the requirements for signal processing performed by the sensor signal processing unit 120 (more specifically, the three-dimensional image generation unit 121 and the biometric information extraction unit 122).

[0032] The high frequency signal generating circuit unit 113 generates a signal for sensing. FIG. 6 is a diagram for explaining the flow of signals in the radio wave transmitting and receiving unit 110 of the radio wave sensor 11 according to the first embodiment. There are various modulation methods for the sensing signal of the radio wave sensor 11, but here we will explain an example in which the FM-CW (Frequency Modulation - Continuous Wave) method, which is often used for automotive applications, is applied as the modulation method. The FM-CW method is suitable for use in occupant monitoring systems because it can measure the distance, speed, and angle of a target object.

[0033] High frequency signal generating circuit section 113 periodically generates an FM signal (called a chirp wave) whose frequency increases and decreases as shown in FIG.

[0034] The transmission circuit unit 114 amplifies the signal power to obtain the power necessary for radiating radio waves, and radiates the radio waves into the space inside the vehicle via the transmission antenna 111 .

[0035] When the radio waves irradiated into the vehicle interior space reach a target object present within the radio wave irradiation range (see "15" in Fig. 5) of the radio wave sensor 11, a portion of the radio waves is reflected by the surface of the target object and returns to the radio wave transmitting / receiving unit 110. Here, the target object refers to an object that reflects radio waves, such as the occupant 30, the occupant 31, or a vehicle structure shown in Fig. 4.

[0036] The receiving circuit section 115 receives, via the receiving antenna 112, the radio waves that have been reflected off the surface of the target object and returned to the radio wave transmitting / receiving section 110. A signal similar to the FM transmission wave is input as a reception signal to the reception circuit unit 115. As shown in FIG. 7A, the reception signal is a time difference t d and input to receiving circuit section 115. Fig. 7A is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave.

[0037] The receiving circuit section 115 calculates the frequency difference f between the frequency of the FM signal generated by the high frequency signal generating circuit section 113 and the frequency of the received signal. d Extract the frequency difference f d and outputs the intermediate frequency signal to an A / D (analog-to-digital) conversion unit 116. FIG. 7B shows the transmission and reception period of the FM transmission wave and the frequency difference f d FIG.

[0038] 4, the occupant 31 is located farther from the radio wave sensor 11 than the occupant 30. Therefore, when the occupant 31 is the target object, the time difference t d and the frequency difference f d is the time difference t when the occupant 30 is the target object. d and the frequency difference f d becomes larger than

[0039] The A / D converter 116 converts the intermediate frequency signal from an analog signal to a digital signal. The A / D conversion unit 116 outputs the digital signal to the sensor signal processing unit 120.

[0040] The sensor signal processing unit 120 performs the following two types of signal processing (hereinafter referred to as "three-dimensional image generation processing" and "biometric information generation processing"). The three-dimensional image generation process is performed by a three-dimensional image generation unit 121 included in the sensor signal processing unit 120. The biometric information generation process is performed by a biometric information extraction unit 122 included in the sensor signal processing unit 120.

[0041] First, the three-dimensional image generation process performed by the three-dimensional image generation unit 121 will be described. The three-dimensional image generating unit 121 generates a three-dimensional image based on the intermediate frequency signal output from the A / D conversion unit 116 of the radio wave transmitting and receiving unit 110 in the three-dimensional image generating process. The intermediate frequency signal is generated from a reflected wave of a radio wave reflected by the surface of the target object. From this intermediate frequency signal, the three-dimensional image generator 121 can calculate the position of the reflection point on the surface of the target object (more specifically, the distance and angle from the radio wave sensor 11). Since there are a plurality of reflection points on the surface of the target object, the three-dimensional image generating unit 121 can generate a three-dimensional image of the target object by performing signal processing on the waves reflected from the plurality of reflection points. The three-dimensional image generated by the three-dimensional image generating unit 121 represents in three dimensions the distribution of areas in the vehicle interior where moving objects exist. In detail, the three-dimensional image represents the minute movements of moving objects in the vehicle interior, in other words, the target objects that have reflected the radio waves irradiated by the radio wave sensor 11, using a plurality of grids that correspond to the reflection points of the radio waves in three-dimensional space.

[0042] 8A, 8B, and 8C are diagrams showing examples of three-dimensional images generated by three-dimensional image generating section 121 in the first embodiment. Fig. 8A shows an example of a top view of a three-dimensional image, Fig. 8B shows an example of a side view of a three-dimensional image, and Fig. 8C shows an example of a front view of a three-dimensional image. Note that Figs. 8A, 8B, and 8C show the three-dimensional image in three-view views for ease of explanation. The three-dimensional images shown in Figs. 8A, 8B, and 8C are three-dimensional images in which the interior of the vehicle is the same as the interior of the vehicle shown in Fig. 4.

[0043] In a three-dimensional image, each grid is assigned a numerical value according to the speed of a target object present at the grid position. Specifically, the faster the speed of a target object present at a grid position, the larger the numerical value assigned to that grid. In other words, in a three-dimensional image, a grid included in a range where a moving object with slight movement exists is assigned a larger numerical value than a grid where no moving object exists. In the three-dimensional images shown in Figures 8A, 8B, and 8C, grids with larger assigned numerical values ​​are shown darker. In other words, in the three-dimensional images shown in Figures 8A, 8B, and 8C, grids that are included in an area where a moving object exists are shown darker. In addition, in a three-dimensional image, stationary objects, in other words, reflected wave components with a velocity of 0, do not appear.

[0044] As shown in Figure 4, the occupant 31, i.e., the infant, is occluded by the child car seat 20 or other vehicle structures. Due to this, the spatial distribution of the infant shown by the grid is barely visible in the 3D images shown in Figures 8A, 8B, and 8C. In other words, it is difficult to determine the presence of an infant from the 3D images shown in Figures 8A, 8B, and 8C. As for the occupant 30, since the radio waves from the radio wave sensor 11 are irradiated onto the entire body, the spatial distribution of the occupant 30 appears on the three-dimensional image as a grid based on the reflected waves.

[0045] Next, the biometric information generation process performed by the biometric information extraction unit 122 will be described. In the biometric information generation process, the biometric information extraction unit 122 extracts, as biometric information, frequency components (Doppler components) superimposed on the reflected waves of a moving object due to minute movements such as breathing of the moving object, based on the intermediate frequency signal output from the A / D conversion unit 116 of the radio wave transmission / reception unit 110. Fig. 9 is a diagram showing Doppler components when a moving object (occupant) is present and when no moving object (occupant) is present. Even if the spatial distribution of an infant does not appear on the three-dimensional image, as in the three-dimensional images shown in Figures 8A, 8B, and 8C, the infant's biological information can be extracted by extracting the Doppler component.

[0046] However, the biometric information does not include information indicating the position of the moving object indicated by the biometric information, and therefore the position of the moving object cannot be identified from the biometric information. That is, even if an attempt is made to determine the presence or absence of an occupant in the vehicle cabin from biometric information extracted based on the reflected waves of radio waves emitted by the radio wave sensor 11 reflected by the surface of a target object, it is difficult to distinguish between an occupant in the vehicle cabin and a person outside the vehicle cabin. For example, if the interior and exterior of the vehicle are as shown in FIG. 5, the occupant 35 outside the vehicle cabin shown in FIG. 5 cannot be identified from the biometric information as a person outside the vehicle cabin. Furthermore, as described above, the radio wave sensor 11 has the property of transmitting, reflecting, or diffracting radio waves depending on the material of the vehicle structure. Due to this property, there is a possibility that the reflected waves that hit the surface of a person outside the vehicle cabin will propagate into the vehicle cabin and then hit the vehicle structure, and then be received by the radio wave sensor 11. In this case, the biometric information extraction unit 122 may extract the biometric information of the person outside the vehicle cabin. When attempting to determine whether or not there is an occupant in the vehicle cabin from this biometric information, the person in the vehicle cabin may be regarded as an occupant, and it may be determined that there is an occupant in the vehicle cabin even though there is no occupant.

[0047] The sensor signal processing unit 120 outputs to the biometric determination device 130 the three-dimensional image generated by the three-dimensional image generating unit 121 in the three-dimensional image generating process and the biometric information extracted by the biometric information extracting unit 122 in the biometric information extracting process.

[0048] An example of the configuration of the biometric determination device 130 will be described. As shown in FIG. 2, the biometric determination device 130 includes a biometric information acquisition unit 1301, a three-dimensional image acquisition unit 1302, a vehicle interior determination unit 131, and a vehicle exterior determination unit 135. The vehicle interior determination unit 131 includes a first vehicle interior determination unit 132 and a second vehicle interior determination unit 133 .

[0049] The biological information acquisition unit 1301 acquires the biological information extracted by the biological information extraction unit 122 of the sensor signal processing unit 120. The biometric information acquisition unit 1301 outputs the acquired biometric information to the second vehicle interior determination unit 133 of the vehicle interior determination unit 131. The second vehicle interior determination unit 133 may have the function of the biological information acquisition unit 1301 .

[0050] The three-dimensional image acquisition unit 1302 acquires the three-dimensional image generated by the three-dimensional image generation unit 121 of the sensor signal processing unit 120. The three-dimensional image acquisition unit 1302 outputs the acquired three-dimensional image to the first vehicle interior determination unit 132 of the vehicle interior determination unit 131 and the vehicle exterior determination unit 135. The first vehicle interior determination unit 132 and the vehicle exterior determination unit 135 may have the function of the three-dimensional image acquisition unit 1302 .

[0051] The vehicle interior determination unit 131 determines whether or not a moving object that is regarded as a child is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 and the biological information acquired by the biological information acquisition unit 1301. As described above, the vehicle interior determination unit 131 includes the first vehicle interior determination unit 132 and the second vehicle interior determination unit 133.

[0052] The first vehicle interior determination unit 132 determines whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as "first vehicle interior determination") based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302. In the first embodiment, the process of the first vehicle interior determination by the first vehicle interior determination unit 132 is also referred to as "first vehicle interior determination process." In the first embodiment, the determination by the first vehicle interior determination unit 132 of "whether or not there is a moving object regarded as an infant in the vehicle interior" more specifically means "whether or not there is a moving object regarded as an infant that has been left behind in the vehicle interior." In other words, the first vehicle interior determination unit 132 determines "whether or not there is only a moving object regarded as an infant in the vehicle interior." The first vehicle interior determination process performed by the first vehicle interior determination unit 132 will now be described in detail.

[0053] The first vehicle interior determination unit 132 first determines whether there is a moving object that can be considered to be an occupant in the vehicle interior in the part of the three-dimensional image that corresponds to the area inside the vehicle interior, and if there is a moving object that can be considered to be an occupant, in which seat position the moving object that can be considered to be an occupant is located. Each grid of the 3D image is associated with a coordinate in the real space inside the vehicle cabin, which is expressed in a 3D coordinate system that represents the real space inside the vehicle cabin. The first vehicle interior determination unit 132 can identify an area of ​​the 3D image that corresponds to the area inside the vehicle cabin. Hereinafter, the 3D image of the portion that corresponds to the area inside the vehicle cabin will be referred to as the "vehicle interior 3D image."

[0054] The first vehicle interior determination unit 132 determines whether the spatial distribution included in the vehicle interior three-dimensional image (for example, the spatial distribution shown by a colored grid in Figure 8) is a spatial distribution corresponding to an occupant (hereinafter referred to as an ``occupant spatial distribution'') based on the shape of the spatial distribution. Any method can be used for making a judgment based on the shape of the spatial distribution. For example, a model that has learned the shape of the occupant spatial distribution can be used to determine whether the spatial distribution contained in a three-dimensional image of the vehicle interior is an occupant spatial distribution. The first vehicle interior determination unit 132 also calculates the relative velocity v corresponding to the spatial distribution included in the vehicle interior three-dimensional image. nis greater than a preset threshold value (hereinafter referred to as a "first velocity determination threshold value"), the spatial distribution may be determined to be an occupant spatial distribution. n The first speed determination threshold value is stored in a location that can be referenced by the first vehicle interior determination unit 132. Even if the occupant is, for example, an infant lying in the child seat 20, there is some movement. If the first vehicle interior determination unit 132 determines that the spatial distribution included in the vehicle interior three-dimensional image is an occupant spatial distribution, it determines that a moving object regarded as an occupant is present in the vehicle interior.

[0055] Furthermore, if the first vehicle interior determination unit 132 determines that the spatial distribution included in the vehicle interior three-dimensional image is an occupant space distribution, the first vehicle interior determination unit 132 calculates a distance R n , incident angle θ x,n and the angle of incidence θ y,n The position of the moving object that is considered to be an occupant in the occupant space distribution is identified from each of the above. n and the incident angle θ with respect to the radio wave sensor 11 x,n and the angle of incidence θ y,n If the distance R from the radio wave sensor 11 is known, the first vehicle interior determination unit 132 can identify the position of the moving object that is regarded as an occupant. n and the incident angle θ with respect to the radio wave sensor 11 x,n and the angle of incidence θ y,n are associated with each other. Here, the first vehicle interior determination unit 132 may determine that the moving object considered to be an occupant is not present, for example, if the position of the moving object considered to be an occupant is in a position that is not expected to be a position where an occupant is present.

[0056] Then, the first vehicle interior determining unit 132 determines whether the detected moving object regarded as an occupant is an adult or a child, based on the occupant spatial distribution included in the vehicle interior three-dimensional image. Any method may be used for the determination based on the size of the occupant space distribution, but for example, a conceivable mode is to determine whether a moving object regarded as an occupant is an adult or an infant based on the size of the occupant space distribution. Also, for example, a conceivable mode is to determine whether a moving object regarded as an occupant is an adult or an infant using a model that has learned the size of the occupant space distribution for adults and infants. When there are a plurality of moving objects that are regarded as occupants, the first vehicle interior determining unit 132 determines whether each of the moving objects is an adult or an infant.

[0057] The first vehicle interior determining unit 132 determines whether or not there is a moving object determined to be a child among the moving objects regarded as occupants. When a moving object determined to be an infant is present among the moving objects regarded as occupants, the first vehicle interior determination unit 132 determines whether or not there is another moving object determined to be an adult in addition to the moving object determined to be an infant. When there is no other moving object determined to be an adult in addition to the moving object determined to be an infant, the first vehicle interior determination unit 132 determines that there is a moving object regarded as an infant in the vehicle interior. On the other hand, if there is a moving object determined to be an adult in addition to a moving object determined to be an infant, or if there is no moving object determined to be an infant among the moving objects considered to be occupants, that is, if it is determined that all moving objects considered to be occupants are adults, the first interior determination unit 132 determines that there are no moving objects considered to be infants in the interior of the vehicle.

[0058] The first vehicle interior determination unit 132 outputs to the abandonment determination device 140 a determination result as to whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as the “first vehicle interior determination result”). The first vehicle interior determination result includes information indicating whether or not there is a moving object that is considered to be a small child inside the vehicle interior.

[0059] The second vehicle interior determination unit 133 determines whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as "second vehicle interior determination") based on the biometric information acquired by the biometric information acquisition unit 1301. In the first embodiment, the process of the second vehicle interior determination by the second vehicle interior determination unit 133 is also referred to as "second vehicle interior determination process." The second vehicle interior determination process performed by the second vehicle interior determination unit 133 will now be described in detail.

[0060] The second vehicle interior determination unit 133 determines from the biological information, in other words, from the extraction result of the Doppler component, whether or not the characteristics of the Doppler component are associated with minute movements such as breathing of a person.

[0061] If it is determined that the characteristics of the Doppler component are associated with minute movements such as a person's breathing, the second vehicle interior determination unit 133 determines that there is a moving object in the vehicle interior that is considered to be an infant. Note that it is not possible to distinguish between adults and infants based on the characteristics of the Doppler component. Here, if the second vehicle interior determination unit 133 determines that the characteristics of the Doppler component are associated with minute movements such as a person's breathing, it determines that the movement is due to an infant's breathing. On the other hand, if it is determined that the characteristics of the Doppler component are not associated with minute movements such as human breathing, the second vehicle interior determination unit 133 determines that there is no moving object that can be regarded as a child in the vehicle interior.

[0062] The second vehicle interior determination unit 133 outputs to the abandonment determination device 140 the determination result as to whether or not a moving object regarded as an infant is present in the vehicle interior (hereinafter referred to as the “second vehicle interior determination result”). The second vehicle interior determination result includes information indicating whether or not there is a moving object that is considered to be a small child inside the vehicle interior.

[0063] In the first embodiment, the first vehicle interior determination and the second vehicle interior determination are also collectively referred to simply as "vehicle interior determination." Furthermore, the first vehicle interior determination result and the second vehicle interior determination result are also collectively referred to simply as "vehicle interior determination result."

[0064] The outside-of-vehicle-compartment determination unit 135 determines whether or not there is a moving object that is considered to be a person outside the vehicle cabin (hereinafter referred to as "outside-vehicle-compartment determination") based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302. In the first embodiment, the process by which the outside-vehicle-compartment determination unit 135 performs the outside-vehicle-compartment determination is also referred to as "outside-vehicle-compartment determination process." The vehicle exterior determination process performed by the vehicle exterior determination unit 135 will now be described in detail.

[0065] The outside-of-vehicle-compartment determining unit 135 determines whether or not a moving object that is regarded as a person is present in a portion of the three-dimensional image that corresponds to the area outside the vehicle compartment. As described above, each grid of the three-dimensional image is associated with a coordinate in the real space inside the vehicle cabin, which is expressed in a three-dimensional coordinate system that represents the real space inside the vehicle cabin, so that the vehicle exterior determination unit 135 can identify an area of ​​the three-dimensional image that corresponds to the area outside the vehicle cabin. Hereinafter, the three-dimensional image of the portion that corresponds to the area outside the vehicle cabin will be referred to as the "vehicle exterior three-dimensional image."

[0066] The outside-of-vehicle determination unit 135 determines whether or not the spatial distribution is a spatial distribution corresponding to a person (hereinafter referred to as a "person spatial distribution") based on the shape of the spatial distribution included in the outside-of-vehicle three-dimensional image. Any method can be used for making a judgment based on the shape of the spatial distribution. For example, a model that has learned the shape of human spatial distribution can be used to determine whether the spatial distribution contained in the 3D image of the exterior of the vehicle is a human spatial distribution. The vehicle exterior determination unit 135 also calculates the relative velocity v corresponding to the spatial distribution included in the vehicle exterior three-dimensional image. n is greater than a preset threshold (hereinafter referred to as a "second speed determination threshold"), the spatial distribution may be determined to be a human spatial distribution. The second speed determination threshold is stored in a location that can be referenced by the vehicle exterior determination unit 135.

[0067] If the outside-of-vehicle-compartment determining unit 135 determines that the spatial distribution included in the outside-of-vehicle-compartment three-dimensional image is a human spatial distribution, it determines that a moving object regarded as a person is present outside the vehicle compartment. On the other hand, if the outside-of-vehicle-compartment determining unit 135 determines that the spatial distribution included in the outside-of-vehicle-compartment three-dimensional image is not a human spatial distribution, it determines that there is no moving object that can be regarded as a person outside the vehicle compartment. Unlike the first interior determination unit 132, the exterior determination unit 135 does not determine the position or size (adult or child) of a moving object that is considered to be a person, but only determines whether or not there is a moving object that is considered to be a person outside the vehicle.

[0068] The outside-vehicle determination unit 135 outputs to the abandonment determination device 140 a determination result as to whether or not there is a moving object that is regarded as a person outside the vehicle (hereinafter referred to as an “outside-vehicle determination result”). The result of the vehicle exterior determination includes information indicating whether or not there is a moving object that can be considered a person outside the vehicle interior.

[0069] An example of the configuration of the abandonment determination device 140 will be described. As shown in FIG. 3, the abandoned vehicle determination device 140 includes a vehicle interior determination result acquisition unit 141, an exterior determination result acquisition unit 142, a communication status information acquisition unit 143, and an abandoned vehicle determination unit 14. The abandonment determination unit 14 includes a first determination unit 144 and a second determination unit 145.

[0070] The vehicle interior determination result acquisition unit 141 acquires the vehicle interior determination result from the vehicle interior determination unit 131 of the biometric determination device 130. More specifically, the vehicle interior determination result acquisition unit 141 acquires the first vehicle interior determination result from the first vehicle interior determination unit 132 of the biometric determination device 130, and acquires the second vehicle interior determination result from the second vehicle interior determination unit 133 of the biometric determination device 130. The vehicle interior determination result acquisition unit 141 outputs the acquired vehicle interior determination result to the abandoned vehicle determination unit 14. The abandoned vehicle determination unit 14 may have the function of the vehicle interior determination result acquisition unit 141.

[0071] The outside-vehicle determination result acquisition unit 142 acquires the outside-vehicle determination result from the outside-vehicle determination unit 135 of the living body determination device 130. The outside-of-vehicle determination result acquisition unit 142 outputs the acquired outside-of-vehicle determination result to the abandoned vehicle determination unit 14. The abandoned vehicle determination unit 14 may have the function of the outside vehicle compartment determination result acquisition unit 142 .

[0072] The communication state information acquisition unit 143 acquires communication state information from the wireless communication state monitoring unit 41 of the vehicle information device 40 . The communication status information acquisition unit 143 outputs the acquired communication status information to the abandonment determination unit 14. The abandonment determination unit 14 may have the function of the communication status information acquisition unit 143 .

[0073] The abandonment determination unit 14 determines whether or not a child has been left behind in the vehicle cabin based on the vehicle interior determination result of the vehicle interior determination performed by the vehicle interior determination unit 131 in the biometric determination device 130, the vehicle exterior determination result of the vehicle exterior determination performed by the vehicle exterior determination unit 135 in the biometric determination device 130, and the communication status information obtained from the wireless communication status monitoring unit 41 of the vehicle information device 40. As described above, the abandonment determination unit 14 includes the first determination unit 144 and the second determination unit 145.

[0074] The first determination unit 144 determines whether or not an infant has been left behind inside the vehicle (hereinafter referred to as "first abandonment determination") based on the vehicle interior determination result of the vehicle interior determination performed by the vehicle interior determination unit 131 and the vehicle exterior determination result of the vehicle exterior determination performed by the vehicle exterior determination unit 135. In the first embodiment, the process by which the first determination unit 144 performs the first abandonment determination is also referred to as "first abandonment determination process."

[0075] In the first embodiment, the determination as to whether or not an infant has been left behind in the vehicle compartment made by the first determination unit 144 is a provisional determination. The actual determination as to whether or not an infant has been left behind in the vehicle compartment is made by the second determination unit 145 in response to the provisional determination result made by the first determination unit 144. Details of the second determination unit 145 will be described later.

[0076] The first abandonment determination process performed by the first determination unit 144 will be described in detail.

[0077] The first determination unit 144 determines whether or not a child has been left behind inside the vehicle based on the vehicle interior determination result and the vehicle exterior determination result in an overlapping manner. For example, the first judgment unit 144 judges whether or not a child has been left behind inside the vehicle cabin based on a table (hereinafter referred to as the "first table for leaving behind judgment") that associates the results of the inside vehicle cabin judgment, the results of the outside vehicle cabin judgment, and information indicating whether or not a child has been left behind inside the vehicle cabin. The first table for abandonment determination is generated in advance by an administrator or the like, and is stored in a location that the first determination unit 144 can refer to.

[0078] FIG. 10 is a diagram showing an example of the first abandonment determination table used by the first determination unit 144 in the first abandonment determination process in the first embodiment. In the first table for abandonment determination shown in Figure 10, "vehicle interior detection / determination result" indicates the vehicle interior detection result, "infant present (using 3D image)" indicates that the first vehicle interior determination result is information that there is a moving object considered to be an infant in the vehicle interior, and "no infant present (using 3D image)" indicates that the first vehicle interior determination result is information that there is no moving object considered to be an infant in the vehicle interior. Also, "infant present (using biological information)" indicates that the second vehicle interior determination result is information that there is a moving object considered to be an infant in the vehicle interior, and "no infant present (using biological information)" indicates that the second vehicle interior determination result is information that there is no moving object considered to be an infant in the vehicle interior. Also, "vehicle exterior detection / determination result" indicates the vehicle exterior detection result. In the first table for determining whether an infant has been left behind shown in FIG. 10, the "result of primary determination of whether an infant has been left behind" indicates information indicating whether an infant has been left behind in the vehicle.

[0079] The first determination unit 144 checks the vehicle interior determination result and the vehicle exterior determination result against a first table for abandonment determination as shown in FIG. 10, and determines whether or not an infant has been left behind inside the vehicle.

[0080] For example, if the first vehicle interior judgment result indicates that there is a moving object considered to be a small child in the vehicle interior, the first judgment unit 144 will judge that a small child has been left behind in the vehicle interior, in other words, that a small child has been left behind in the vehicle interior, regardless of the content of the second vehicle interior judgment result. In addition, the first vehicle interior judgment result being a judgment result indicating that a moving object deemed to be a small child is present in the vehicle interior means, in other words, that the first vehicle interior judgment unit 132 in the biometric judgment device 130 has determined in the first vehicle interior judgment that a moving object deemed to be a small child is present in the vehicle interior.

[0081] The determination of whether or not a moving object deemed to be an infant is present inside the vehicle cabin based on the three-dimensional image of the vehicle cabin, i.e., the first vehicle cabin determination, has a high degree of reliability. That is, the reliability of the first vehicle cabin determination result is high. Therefore, if the first vehicle cabin determination determines that a moving object deemed to be an infant is present inside the vehicle cabin, the first determination unit 144 can determine that an infant has been left inside the vehicle cabin, regardless of the second vehicle cabin determination result and the vehicle exterior determination result.

[0082] Furthermore, for example, if the first vehicle interior determination result indicates that there is no moving object considered to be an infant inside the vehicle interior, the second vehicle interior determination result indicates that there is a moving object considered to be an infant inside the vehicle interior, and the vehicle exterior determination result indicates that there is a moving object considered to be a person outside the vehicle interior, the first determination unit 144 determines that it is indeterminate whether or not an infant has been left inside the vehicle interior, in other words, whether or not an infant has been left inside the vehicle interior. In other words, the first determination unit 144 sets the first abandonment determination to "suspend." In other words, the first determination unit 144 does not determine whether or not an infant has been left inside the vehicle interior. In addition, the first vehicle interior judgment result being a judgment result indicating that there is no moving object considered to be a small child in the vehicle interior means, in other words, that the first vehicle interior judgment unit 132 in the biometric judgment device 130 has determined in the first vehicle interior judgment that there is no moving object considered to be a small child in the vehicle interior. The second vehicle interior determination result being a determination result indicating that there is a moving object considered to be a small child in the vehicle interior means, in other words, that the second vehicle interior determination unit 133 in the biometric determination device 130 has determined in the second vehicle interior determination that there is a moving object considered to be a small child in the vehicle interior. The outside-vehicle determination result indicating that there is a moving object that can be considered to be a person outside the vehicle cabin means, in other words, that the outside-vehicle determination unit 135 in the biometric determination device 130 has determined that there is a moving object that can be considered to be a person outside the vehicle cabin in the outside-vehicle determination.

[0083] 8, in the first vehicle interior determination based on the vehicle interior three-dimensional image, even if a moving object deemed to be an infant is present, it may not be possible to determine this due to the influence of a child seat or vehicle structures blocking radio waves from the radio wave sensor 11. Therefore, when it is determined in the first vehicle interior determination that no moving object deemed to be an infant is present, the first determination unit 144 utilizes the second vehicle interior determination result of the second vehicle interior determination based on biometric information. As a result, if it is determined in the second vehicle interior determination that a moving object regarded as a child is present in the vehicle interior, the first determination unit 144 further utilizes the result of the vehicle exterior determination.

[0084] As described above, even if an attempt is made to determine the presence or absence of an occupant in the vehicle cabin from biometric information, it is difficult to distinguish between an occupant in the vehicle cabin and a person outside the vehicle cabin. Furthermore, as described above, due to the property of the radio waves emitted by the radio wave sensor 11 to transmit, reflect, or diffract depending on the material of the vehicle structure, there is a possibility that the reflected waves that hit the surface of a person outside the vehicle cabin will propagate into the vehicle cabin and further hit a structure inside the vehicle cabin, and then be received by the radio wave sensor 11. Therefore, when an attempt is made to determine the presence or absence of an occupant in the vehicle cabin from biometric information, the person inside the vehicle cabin may be regarded as an occupant in the vehicle cabin, and it may be determined as if there is an occupant in the vehicle cabin even though there is no occupant in the vehicle cabin. Therefore, even if the second vehicle interior determination determines that a moving object deemed to be a child is present inside the vehicle interior, if the vehicle exterior determination determines that a moving object deemed to be a person is present outside the vehicle interior, the moving object deemed to be a child that was determined to be present inside the vehicle interior may not actually be present inside the vehicle interior. In other words, the second vehicle interior determination unit 133 may have made an erroneous determination in the second vehicle interior determination process. If the first determination unit 144 determines that an infant has been left inside the vehicle interior, this determination may also be an erroneous determination. If the first determination unit 144 determines that an infant has been left inside the vehicle interior, i.e., the first abandonment determination, is an erroneous determination, the alarm control unit 150 may output a false alarm in the subsequent alarm output control by the alarm control unit 150.

[0085] Therefore, if the first vehicle interior judgment result is a judgment result indicating that there is no moving object considered to be an infant inside the vehicle interior, the second vehicle interior judgment result is a judgment result indicating that there is a moving object considered to be an infant inside the vehicle interior, and the outside vehicle interior judgment result is a judgment result indicating that there is a moving object considered to be a person outside the vehicle interior, the first judgment unit 144 considers this combination of judgment results to be one that may be an erroneous judgment, sets a status that makes the judgment result of the first abandonment judgment as to whether or not an infant has been left inside the vehicle interior indeterminate, puts the first abandonment judgment on hold, and suppresses erroneous judgments of the first abandonment judgment.

[0086] On the other hand, for example, if the first vehicle interior determination result is a determination result indicating that there is no moving object considered to be a small child inside the vehicle interior, the second vehicle interior determination result is a determination result indicating that there is a moving object considered to be a small child inside the vehicle interior, and the vehicle exterior determination result is a determination result indicating that there is no moving object considered to be a person outside the vehicle interior, the first determination unit 144 determines that an infant has been left behind inside the vehicle interior, in other words, that an infant has been left behind inside the vehicle interior. The outside-cabin determination result indicating that there is no moving object that can be considered a person outside the vehicle cabin means, in other words, that the outside-cabin determination unit 135 in the biometric determination device 130 has determined that there is no moving object that can be considered a person outside the vehicle cabin in the outside-cabin determination.

[0087] As described above, even if the second interior determination determines that there is a moving object considered to be a child inside the vehicle, if the exterior determination determines that there is a moving object considered to be a person outside the vehicle, it is possible that the moving object considered to be a child that was determined to be inside the vehicle is actually a moving object that is not inside the vehicle. However, in the above example, since it is determined in the outside vehicle compartment determination that there is no moving object that is considered to be a person outside the vehicle compartment, it is unlikely that the moving object that is considered to be a child and that is determined to be present inside the vehicle compartment in the second inside vehicle compartment determination is actually a moving object that is not present inside the vehicle compartment. Therefore, the first determination unit 144 can determine that there is no case of an infant being left behind inside the vehicle compartment.

[0088] Furthermore, for example, if the first vehicle interior determination result is a determination result indicating that there is no moving object considered to be a small child inside the vehicle interior, and the second vehicle interior determination result is a determination result indicating that there is no moving object considered to be a small child inside the vehicle interior, the first determination unit 144 will ignore the vehicle exterior determination result and determine that no small child has been left inside the vehicle interior, in other words, that no small child has been left inside the vehicle interior. In addition, the second vehicle interior determination result being a determination result indicating that there is no moving object considered to be a small child in the vehicle interior means, in other words, that the second vehicle interior determination unit 133 in the biometric determination device 130 has determined in the second vehicle interior determination that there is no moving object considered to be a small child in the vehicle interior.

[0089] When it is determined in the first vehicle interior determination that there is no moving object that is deemed to be an infant, the first determination unit 144 utilizes the second vehicle interior determination result of the second vehicle interior determination based on biological information. As a result, if the second interior judgment determines that there is no moving object considered to be a child inside the vehicle, the first judgment unit 144 can determine that no child has been left behind inside the vehicle, regardless of the result of the exterior judgment.

[0090] In this way, the first determination unit 144 determines whether or not an infant has been left inside the vehicle compartment based on, for example, the vehicle interior determination result, the vehicle exterior determination result, and the first table for abandonment determination. The contents of the first table for abandonment determination shown in Fig. 10 are merely an example. The first abandonment determination result for the combination of the vehicle interior determination result and the vehicle exterior determination result can be set appropriately based on the intention (policy) of a designer such as an administrator.

[0091] Furthermore, the above-described method of first abandonment determination by the first determination unit 144 is merely one example. For example, the first judgment unit 144 may perform the first abandonment judgment by inputting the vehicle interior judgment result output from the vehicle interior judgment result acquisition unit 141 and the vehicle exterior judgment result output from the vehicle exterior judgment result acquisition unit 142 into a trained model that receives the vehicle interior judgment result and the vehicle exterior judgment result as input and outputs information indicating whether or not an infant has been left behind inside the vehicle, and obtaining information indicating whether or not an infant has been left behind inside the vehicle.

[0092] The first determination unit 144 outputs to the second determination unit 145 the determination result as to whether or not an infant has been left behind in the vehicle compartment (hereinafter referred to as the “first abandonment determination result”). The first abandonment determination result includes information indicating whether or not a child has been left behind in the vehicle compartment, or whether or not it is uncertain (pending).

[0093] The second determination unit 145 determines whether or not an infant has been left behind in the vehicle compartment (hereinafter referred to as "second abandonment determination") based on the first abandonment determination result of the first abandonment determination performed by the first determination unit 144 and the communication status information acquired by the communication status information acquisition unit 143. In the first embodiment, the process by which the second determination unit 145 performs the second abandonment determination is also referred to as "second abandonment determination process."

[0094] In addition, in embodiment 1, the determination made by second determination unit 145 as to whether or not a child has been left behind in the vehicle cabin is a main determination that finalizes the determination result as to whether or not a child has been left behind in the vehicle cabin.

[0095] The second abandonment determination process performed by the second determination unit 145 will be described in detail.

[0096] The second determination unit 145 determines whether or not an infant has been left behind in the vehicle compartment based on the first abandonment determination result and the communication status information in a redundant manner.

[0097] For example, the second judgment unit 145 judges whether or not a child has been left behind in the vehicle cabin based on a table (hereinafter referred to as the "second table for leaving behind judgment") in which the first leaving behind judgment result, information indicating whether wireless communication with the short-range wireless communication device 45 has been established, and information indicating whether or not a child has been left behind in the vehicle cabin are associated. The second table for abandonment determination is generated in advance by an administrator or the like, and is stored in a location that the second determination unit 145 can refer to.

[0098] FIG. 11 is a diagram showing an example of the second abandonment determination table used by the second determination unit 145 in the second abandonment determination process in the first embodiment. In the second table for determining whether an infant has been left behind shown in Figure 11, the "primary infant abandonment determination result" indicates the first infant abandonment determination result, the "wireless communication status" indicates whether wireless communication has been established between the vehicle and the short-range wireless communication device 45, and the "secondary infant abandonment determination result" indicates information indicating whether an infant has been left behind in the vehicle cabin.

[0099] The second determination unit 145 checks the first abandonment determination result and the communication status information against a second abandonment determination table as shown in FIG. 11 to determine whether or not an infant has been left behind in the vehicle. The second determination unit 145 can determine whether or not wireless communication with the short-range wireless communication device 45 has been established from the communication state information.

[0100] When wireless communication between the vehicle and the short-range wireless communication device 45 is established, in other words, when a person is in the vicinity of the vehicle, the second judgment unit 145 assumes, for example, that the vehicle owner is in the vicinity of the vehicle and would be able to recognize even if there is a small child inside the vehicle, and judges that no small child has been left inside the vehicle, in other words, that no small child has been left inside the vehicle.

[0101] For example, if the first abandonment judgment result is "infant abandoned," or if the first abandonment judgment result is "infant abandoned (pending)," and wireless communication is established between the vehicle and the short-range wireless communication device 45, the second judgment unit 145 judges that an infant has not been left behind in the vehicle cabin. The first abandonment determination result being "infant abandoned" means, in other words, that the first abandonment determination unit 14 has determined in the first abandonment determination that an infant has been abandoned inside the vehicle. The first abandonment determination result being "infant abandonment uncertain (pending)" means, in other words, that the first abandonment determination unit 14 has put the first abandonment determination on hold.

[0102] For example, if the first abandonment judgment result is "infant abandoned," or if the first abandonment judgment result is "infant abandoned (pending)," and wireless communication between the vehicle and the short-range wireless communication device 45 has not been established, the second judgment unit 145 judges that an infant has been left behind in the vehicle cabin. In this case, the second determination unit 145 assumes, for example, that the owner of the vehicle has left the vehicle without realizing that a small child has been left inside the vehicle.

[0103] In this way, the second determination unit 145 determines whether or not an infant has been left behind in the vehicle compartment based on, for example, the first abandonment determination result, the communication status information, and the second abandonment determination table. 11 is merely an example. The second abandonment determination result for the combination of the first abandonment determination result and whether wireless communication between the vehicle and the short-range wireless communication device 45 is established or not can be set appropriately based on the intention (policy) of a designer such as an administrator.

[0104] Furthermore, the above-described method of second abandonment determination by the second determination unit 145 is merely an example. For example, the second judgment unit 145 may perform a second abandonment judgment by inputting the first abandonment judgment result output from the first judgment unit 144 and the communication status information output from the communication status information acquisition unit 143 into a trained model that receives the first abandonment judgment result and communication status information as input and outputs information indicating whether or not an infant has been left behind in the vehicle cabin, and obtaining information indicating whether or not an infant has been left behind in the vehicle cabin.

[0105] The second determination unit 145 outputs to the alarm control device 150 the determination result as to whether or not an infant has been left behind in the vehicle compartment (hereinafter referred to as the "second abandonment determination result"). The second abandonment determination result includes information indicating whether or not a small child has been left behind in the vehicle. The second determination unit 145 outputs the second abandonment determination result to the alarm control device 150 together with the first abandonment determination result. In the first embodiment, the first abandonment determination result and the second abandonment determination result are also collectively referred to simply as "abandonment determination result."

[0106] An example of the configuration of the alarm control device 150 will be described. As shown in FIG. 1, the alarm control device 150 includes an abandonment determination result acquisition unit 151 and an alarm control unit 152.

[0107] The abandonment determination result acquisition unit 151 acquires the abandonment determination result from the abandonment determination device 140. More specifically, the abandonment determination result acquisition unit 151 acquires the first abandonment determination result and the second determination result from the abandonment determination device 140. The abandonment determination result acquisition unit 151 outputs the acquired abandonment determination result to the alarm control unit 152.

[0108] When the abandonment determination device 140 determines that an infant has been left behind in the vehicle cabin, more specifically, when the first determination unit 144 provisionally determines that an infant has been left behind in the vehicle cabin, or when the second determination unit 145 determines that an infant has been left behind in the vehicle cabin, the alarm control unit 152 outputs an instruction (hereinafter referred to as an "alarm instruction") to the alarm device 160 to output an alarm.

[0109] The warning control unit 152 can select whether to output a warning instruction based on the first abandonment determination result or based on the second abandonment determination result. In other words, the alarm control unit 152 can select whether to output an alarm instruction based on the result of the first abandonment judgment by the first judgment unit 144, in other words, a provisional judgment, or whether to output an alarm instruction based on the result of the second abandonment judgment by the second judgment unit 145, in other words, a final judgment.

[0110] For example, the alarm control unit 152 determines whether to output an alarm instruction based on a table in which the first abandonment judgment result is associated with information indicating whether an alarm should be output (hereinafter referred to as the "first alarm table"), or a table in which the second abandonment judgment result is associated with information indicating the infant for whom an alarm should be output (hereinafter referred to as the "second alarm table"). The first alarm table and the second alarm table are generated in advance by an administrator or the like, and are stored in a location that the alarm control unit 152 can refer to.

[0111] When it is selected to output a warning instruction based on the first abandonment determination result, the warning control unit 152 uses the first table for warning to determine whether or not a warning needs to be output. When it is selected to output a warning instruction based on the second abandonment determination result, the warning control unit 152 uses the second warning table to determine whether or not a warning output is necessary.

[0112] FIG. 12 is a diagram showing an example of the first alarm table and the second alarm table used by the alarm control unit 152 in determining whether to output an alarm instruction in the first embodiment. FIG. 12A shows an example of the first alarm table, and FIG. 12B shows an example of the second alarm table. In the first table for alarm shown in FIG. 12A, "Yes", "Indeterminate", and "No" indicate the first abandonment determination result, and "Alarm ON" and "Alarm OFF" indicate whether an alarm needs to be output. In the second table for warning shown in FIG. 12B, "Yes" and "No" indicate the second abandonment determination result, and "Warning ON" and "Warning OFF" indicate whether or not a warning is required.

[0113] For example, when selecting to output an alarm instruction based on the first abandonment determination result, if the first abandonment determination result indicates that an infant has been left in the vehicle (pending), in other words, if the first determination unit 144 has decided to suspend the first abandonment determination, the alarm control unit 152 determines that an alarm output is unnecessary (alarm OFF). In this case, the alarm control unit 152 does not output an alarm instruction. As described above, the first determination unit 144 suspends the first abandonment determination when there is a possibility that the first abandonment determination is an erroneous determination. The alarm control unit 152 determines that an alarm output is unnecessary in order to eliminate the annoyance of a false alarm due to an erroneous determination of the first abandonment determination, that is, an erroneous determination of whether or not a child has been left behind in the vehicle compartment.

[0114] For example, when selecting to output an alarm instruction based on the second abandonment determination result, if the second abandonment determination result indicates that an infant has been left inside the vehicle, in other words, if the second determination unit 145 determines that an infant has been left inside the vehicle, the alarm control unit 152 determines that an alarm needs to be output (alarm ON). In this case, the alarm control unit 152 outputs an alarm instruction.

[0115] For example, the alarm control unit 152 may output an alarm instruction when a state in which it has been determined that an alarm output is necessary continues for a predetermined time period (hereinafter referred to as the "alarm determination time period") or longer. The alarm determination time period is, for example, 10 seconds. Note that this is merely an example, and the alarm determination time period can be set to any appropriate length.

[0116] The alarm device 160 outputs an alarm based on the control of the alarm control device 150 . The alarm device 160 outputs an alarm when an alarm instruction is output from the alarm control device 150. For example, if the alarm device 160 is a mobile terminal carried by the owner of the vehicle, the alarm device 160 displays a message indicating that an object has been left inside the vehicle. The alarm device 160 may output a buzzer, or may output a voice message indicating that an object has been left inside the vehicle. For example, if the alarm device 160 is a horn provided in the vehicle, the alarm device 160 sounds. to If so, the alarm 160 will flash.

[0117] It is problematic in practice if the system determines that a child has been left behind in the vehicle even though the child is not, and issues a false alarm to the vehicle owner or people around the vehicle. As described above, the first determination unit 144 in the abandonment determination device 140 suspends the first abandonment determination if there is a possibility that the first abandonment determination is an erroneous determination, thereby suppressing erroneous determinations that an infant has been left behind in the vehicle compartment, and the alarm control unit 152 in the alarm control device 150 does not output an alarm instruction based on the first abandonment determination result that indicates that the first abandonment determination has been suspended, thereby suppressing false alarms that an infant has been left behind in the vehicle compartment. In this way, the abandonment determination system 100 can solve the above practical problems.

[0118] Furthermore, the second determination unit 145 in the abandonment determination device 140 performs a second abandonment determination by combining the 3D image and biometric information output from the radio wave sensor 11 with the communication status information, thereby preventing the second determination unit 145 from making a false determination that an infant has been abandoned in the vehicle cabin and from outputting unnecessary alarms to the vehicle owner, etc., which could be considered as false alarms. The alarm control unit 152 in the alarm control device 150 controls the output of an alarm instruction based on the second abandonment determination result, which takes into account the communication status between the vehicle and the short-range wireless communication device 45, thereby preventing false alarms that an infant has been abandoned in the vehicle cabin and from outputting unnecessary alarms to the vehicle owner, etc., which could be considered as false alarms. In this way, the abandonment determination system 100 can prevent false alarms that an infant has been abandoned in the vehicle cabin and from outputting unnecessary alarms to the vehicle owner, etc.

[0119] The operation of the abandonment determination system 100 according to the first embodiment will be described. FIG. 13 is a flowchart for explaining the operation of the abandonment determination system 100 according to the first embodiment. The abandonment determination system 100 repeats the operations shown in the flowchart of Figure 13, for example, when the vehicle doors are locked, the vehicle power is turned off, or the shift position in the vehicle is set to "P," until the vehicle doors are unlocked or the vehicle power is turned on. In FIG. 13, the processes of step ST1-1, and steps ST1-2 to ST3 are performed by the living body determination device 130, and the processes of step ST4-1, step ST4-2, and steps ST4-3 to ST6 are performed by the abandonment determination device 140.

[0120] The biological information acquiring unit 1301 acquires the biological information extracted by the biological information extracting unit 122 of the sensor signal processing unit 120 (step ST1-1). The biometric information acquisition unit 1301 outputs the acquired biometric information to the second vehicle interior determination unit 133 of the vehicle interior determination unit 131.

[0121] The three-dimensional image acquisition unit 1302 acquires the three-dimensional image generated by the three-dimensional image generation unit 121 of the sensor signal processing unit 120 (step ST1-2). The three-dimensional image acquisition unit 1302 outputs the acquired three-dimensional image to the first vehicle interior determination unit 132 of the vehicle interior determination unit 131 and the vehicle exterior determination unit 135.

[0122] The vehicle interior determination unit 131 executes a vehicle interior determination process to determine whether or not a moving object that can be considered a child is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 and the biometric information acquired by the biometric information acquisition unit 1301 in step ST1-1 (step ST2). The vehicle interior determination unit 131 outputs the vehicle interior determination result to the abandoned vehicle determination device 140.

[0123] The outside-of-vehicle determination unit 135 executes outside-of-vehicle determination processing for determining whether the vehicle is outside the vehicle, based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 (step ST3). The outside-of-vehicle determination unit 135 outputs the outside-of-vehicle determination result to the abandoned vehicle determination device 140.

[0124] The vehicle interior determination result acquisition unit 141 acquires the vehicle interior determination result output from the vehicle interior determination unit 131 (step ST4-1). The vehicle interior determination result acquisition unit 141 outputs the acquired vehicle interior determination result to the abandoned vehicle determination unit 14.

[0125] The outside-of-vehicle determination result acquisition unit 142 acquires the outside-of-vehicle determination result from the outside-vehicle determination unit 135 (step ST4-2). The outside-of-vehicle determination result acquisition unit 142 outputs the acquired outside-of-vehicle determination result to the abandoned vehicle determination unit 14.

[0126] The communication state information acquisition unit 143 acquires communication state information from the wireless communication state monitoring unit 41 of the vehicle information device 40 (step ST4-3). The communication status information acquisition unit 143 outputs the acquired communication status information to the abandonment determination unit 14.

[0127] The first judgment unit 144 of the abandonment judgment unit 14 performs a first abandonment judgment process to make a first abandonment judgment based on the vehicle interior judgment result acquired by the vehicle interior judgment result acquisition unit 141 in step ST4-1 and the vehicle exterior judgment result acquired by the vehicle exterior judgment unit in step ST4-2 (step ST5). The first determination unit 144 outputs the first abandonment determination result to the second determination unit 145.

[0128] The second judgment unit 145 of the abandonment judgment unit 14 performs a second abandonment judgment process to make a second abandonment judgment based on the first abandonment judgment result of the first abandonment judgment performed by the first judgment unit 144 in step ST5 and the communication status information acquired by the communication status information acquisition unit 143 in step ST4-3 (step ST6). The second determination unit 145 outputs the abandonment determination result to the alarm control device 150. In detail, the second determination unit 145 outputs the second abandonment determination result to the alarm control device 150 together with the first abandonment determination result.

[0129] 13, the process of step ST1-1 and the process of step ST1-2 are performed in parallel, but this is merely an example. The order of the process of step ST1-1 and the process of step ST1-2 may be reversed. 13, the processing of step ST4-1, the processing of step ST4-2, and the processing of step ST4-3 are performed in parallel, but this is merely an example. The order of the processing of step ST4-1, the processing of step ST4-2, and the processing of step ST4-3 may be reversed.

[0130] FIG. 14 is a flowchart for explaining in detail the processing of step ST2 in FIG. The first vehicle interior determination unit 132 executes a first vehicle interior determination process to make a first vehicle interior determination as to whether or not a moving object that can be considered to be a child is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302 in step ST1-2 of Figure 13 (step ST1401). The first vehicle interior determination unit 132 outputs the first vehicle interior determination result to the abandoned vehicle determination device 140.

[0131] The second vehicle interior determination unit 133 executes a second vehicle interior determination process to make a second vehicle interior determination as to whether or not there is a moving object considered to be a child in the vehicle interior based on the biometric information acquired by the biometric information acquisition unit 1301 in step ST1-1 of Figure 13 (step ST1402). The second vehicle interior determination unit 133 outputs the second vehicle interior determination result to the abandoned vehicle determination device 140.

[0132] 14, the process of step ST1401 is performed first, followed by the process of step ST1402, but this is merely an example. The process of step ST1401 and the process of step ST1402 may be performed in parallel, or the order of the process of step ST1401 and the process of step ST1402 may be reversed.

[0133] In this way, the abandonment determination system 100 performs a vehicle interior determination as to whether or not a moving object deemed to be an infant is present inside the vehicle, based on the three-dimensional image generated by the radio wave sensor 11 and the biological information extracted by the radio wave sensor 11. The abandonment determination system 100 also performs a vehicle exterior determination as to whether or not a moving object deemed to be a person is present outside the vehicle, based on the three-dimensional image. The abandonment determination system 100 then determines whether or not an infant has been left inside the vehicle, based on the determination result of the vehicle interior determination (vehicle interior determination result) and the determination result of the vehicle exterior determination (vehicle exterior determination result). Therefore, the abandonment determination system 100 can prevent erroneous determination of whether or not an infant has been left behind in the vehicle.

[0134] The operation of the alarm control device 150 according to the first embodiment will be described. FIG. 15 is a flowchart for explaining the operation of the alarm control device 150 according to the first embodiment. In the abandonment determination system 100, when an alarm instruction is output from the abandonment determination device 140, the alarm control device 150 performs the operation shown in the flowchart of FIG.

[0135] The abandonment determination result acquisition unit 151 acquires the abandonment determination result from the abandonment determination device 140 (step ST101). More specifically, the abandonment determination result acquisition unit 151 acquires the first abandonment determination result and the second determination result from the abandonment determination device 140. The abandonment determination result acquisition unit 151 outputs the acquired abandonment determination result to the alarm control unit 152.

[0136] The alarm control unit 152 outputs an alarm instruction to the alarm device 160 when the first judgment unit 144 of the abandonment judgment device 140 provisionally judges that an infant has been left behind in the vehicle cabin, or when the second judgment unit 145 judges that an infant has been left behind in the vehicle cabin, based on the abandonment judgment result acquired by the abandonment judgment result acquisition unit 151 in step ST101.

[0137] In this way, the alarm control device 150 outputs an alarm instruction to output an alarm when the abandonment determination device 140 determines that an infant has been left inside the vehicle. This allows the alarm control device 150 to notify the owner of the vehicle or people around the vehicle that an infant has been left inside the vehicle.

[0138] Furthermore, the alarm control device 150 does not output an alarm instruction when the abandonment determination device 140 reserves the determination of whether or not a small child has been left behind in the vehicle compartment. Therefore, the alarm control device 150 can suppress false alarms when, for example, it is erroneously determined that an infant has been left behind in the vehicle compartment even though the infant has not been left behind.

[0139] In addition, the alarm control device 150 can suppress false alarms that a child has been left behind in the vehicle cabin by controlling the output of an alarm instruction based on the determination result (second abandonment determination result) of whether or not a child has been left behind in the vehicle cabin, which the abandonment determination device 140 makes by combining the three-dimensional image and biometric information output from the radio wave sensor 11 with the communication status information. For example, even if only a small child is present in the vehicle, if the vehicle owner is outside the vehicle and wireless communication is established between the vehicle and the short-range wireless communication device 45 carried by the vehicle owner, it is assumed that the owner is aware of the small child inside the vehicle. In this case, an alarm would be unnecessary, and even if an alarm were to be output, it would be considered a false alarm. The alarm control device 150 can suppress such false alarms.

[0140] In the above-described first embodiment, when the first judgment unit 144 of the abandonment judgment device 140 judges to suspend the judgment of whether abandonment has occurred (first abandonment judgment), if the suspended state continues for a predetermined time (hereinafter referred to as the "first suspension judgment time") or more, it may be configured to judge that an infant has been left behind in the vehicle compartment, i.e., that an infant has been left behind in the vehicle compartment. For example, even if the vehicle owner is outside the vehicle cabin and within a distance where the short-range wireless communication device 45 carried by the vehicle owner can establish communication with the vehicle, it is conceivable that for some reason, a small child may enter the vehicle cabin without the owner's knowledge and be unable to get out. Therefore, when the first determination unit 144 determines to suspend the determination of whether or not an infant has been left behind (first abandonment determination), if the suspension state continues for a first suspension determination time or longer, it may determine that an infant has been left behind in the vehicle cabin. The first suspension determination time is, for example, 10 minutes. Note that this is merely an example, and the first suspension determination time may be any time longer than the warning determination time. In this case, for example, the first judgment unit 144 may notify the second judgment unit 145 that it has changed from a state in which the first abandonment judgment result was pending to a judgment that an infant has been left behind, and the second judgment unit 145 may determine that an infant has been left behind in the vehicle regardless of the content of the communication status information. As a result, the abandonment determination system 100 will suspend the determination that a child has been left behind in the vehicle if there is a possibility that the determination is incorrect, and will not output an alarm based on the incorrect determination, thereby preventing a situation in which a truly necessary alarm is not output.

[0141] Furthermore, in the above-described first embodiment, the alarm control unit 152 of the alarm control device 150 may be configured to output an alarm instruction when the first determination unit 144 of the abandonment determination device 140 has suspended the determination of whether abandonment has occurred (first abandonment determination) for a predetermined time period (hereinafter referred to as the "second suspension determination time") or more. The second suspension determination time period is, for example, 10 minutes. Note that this is merely an example, and the second suspension determination time period may be any time period longer than the alarm determination time period. This allows the alarm control device 150 to prevent situations in which truly necessary alarms are not output, such as when, for example, even if the vehicle owner is outside the vehicle cabin and is close enough that the short-range wireless communication device 45 carried by the owner can establish communication with the vehicle, a small child enters the vehicle cabin for some reason without the owner's knowledge and is unable to get out.

[0142] Furthermore, in the above-described embodiment 1, in the alarm control device 150, the alarm control unit 152 can select whether to output an alarm instruction based on the first abandonment determination result or the second abandonment determination result, but this is merely one example. For example, the warning control unit 152 may always output a warning instruction based on the second abandonment determination result, in which case the first warning table does not need to be provided.

[0143] Furthermore, in the above-described first embodiment, the alarm control unit 152 of the alarm control device 150 may, for example, perform control to change the alarm means of the alarm to be output from the alarm device 160 as the time elapses since it is determined that an infant has been left inside the vehicle compartment. For example, the alarm control unit 152 may control the alarm device 160 to output a louder sound as the time elapses since it is determined that an infant has been left inside the vehicle compartment increases. Furthermore, for example, the alarm control unit 152 may control the content of the alarm to be output from the alarm device 160 to change as the time elapses since it is determined that an infant has been left inside the vehicle compartment increases, such as by changing from sending a message to outputting an alarm sound. In this case, for example, the first alarm table or the second alarm table contains information about the alarm means by which the alarm device 160 should output an alarm.

[0144] In the first embodiment, the abandonment determination device 140 includes the first determination unit 144 and the second determination unit 145, but this is merely an example. The abandonment determination device 140 does not necessarily have to include the second determination unit 145, and may be configured without the second determination unit 145. In this case, the first abandonment determination performed by the first determination unit 144 in the abandonment determination device 140 to determine whether or not an infant has been left in the vehicle compartment becomes the main determination. In this case, the alarm control unit 152 in the alarm control device 150 uses the first alarm table to determine whether or not an alarm needs to be output. The first alarm table does not need to be provided. In this case, the processing of step ST6 in the operation of the abandonment determination system 100 described using the flowchart of FIG. 13 can be omitted.

[0145] Furthermore, in the above-described embodiment 1, the determination made by the first vehicle interior determination unit 132 in the vehicle interior determination as to whether or not there is a moving object that can be considered to be a small child in the vehicle interior means whether or not there is only a moving object that can be considered to be a small child in the vehicle interior, but this is merely one example. For example, in the above embodiment 1, the determination made by the first vehicle interior determination unit 132 in the first vehicle interior determination, "whether or not there is a moving object considered to be a small child in the vehicle interior," may also mean "whether or not there is a moving object considered to be a small child in the vehicle interior, regardless of whether or not there are other adult occupants." In this case, for example, the first vehicle interior determination unit 132 outputs, in the vehicle interior determination process, information that associates all moving objects that are considered to be occupants present in the vehicle interior with information indicating whether the moving objects are adults or children to the abandoned vehicle determination device 140 as the first vehicle interior determination result. The first determination unit 144 of the abandonment determination device 140 determines whether or not there are only moving objects that are considered to be small children in the vehicle cabin based on the first vehicle cabin determination result, and uses this determination result as the first vehicle cabin determination result to match with the first table for abandonment determination.

[0146] In the first embodiment, the radio wave sensor 11 may generate a three-dimensional image of the interior of the vehicle and a three-dimensional image of the exterior of the vehicle, respectively. In this case, in the biometric determination device 130, the first vehicle interior determination unit 132 performs a first vehicle interior determination process based on the vehicle interior three-dimensional image, and the vehicle exterior determination unit 135 performs a vehicle exterior determination process based on the vehicle exterior three-dimensional image.

[0147] In the first embodiment, the biometric determination device 130 is mounted on the radio wave sensor 11, but this is merely an example. The biometric determination device 130 may be provided outside the radio wave sensor 11 so as to be connectable to the radio wave sensor 11. For example, the biometric determination device 130 may be installed in the abandonment determination device 140. Furthermore, the present invention is not limited to this, and for example, the abandonment determination device 140 and the alarm control device 150 may be mounted on the radio wave sensor 11. Furthermore, for example, the sensor signal processing unit 120 may be provided outside the radio wave sensor 11. The sensor signal processing unit 120, the living body determination device 130, the abandonment determination device 140, and the alarm control device 150 can be appropriately combined into one device.

[0148] In addition, in the above-mentioned embodiment 1, the components provided in the sensor signal processing unit 120 (biometric information extraction unit 122 and three-dimensional image generation unit 121), the components provided in the biometric determination device 130 (biometric information acquisition unit 1301, three-dimensional image acquisition unit 1302, vehicle interior determination unit 131, and vehicle exterior determination unit 135), the components provided in the abandonment determination device 140 (vehicle interior determination result acquisition unit 141, vehicle exterior determination result acquisition unit 142, communication status information acquisition unit 143, and abandonment determination unit 14), and the components provided in the alarm control device 150 (abandonment determination result acquisition unit 151 and alarm control unit 152) can be appropriately combined to be provided in a single device. For example, the outside vehicle compartment determination unit 135 may be provided in the abandonment determination device 140, or the alarm control unit 152 may be provided in the abandonment determination device 140. Also, for example, the abandonment determination unit 14 may be provided in the living body determination device 130.

[0149] In the first embodiment, the abandonment determination system 100 is configured by the biometric determination device 130 and the abandonment determination device 140, but this is merely an example. For example, the alarm control device 150 may be provided in the abandonment determination system 100, or the sensor signal processing unit 120 may be provided in the abandonment determination system 100.

[0150] In the first embodiment, the living body determination device 130, the abandonment determination device 140, and the alarm control device 150 are in-vehicle devices mounted on a vehicle, but this is merely an example. For example, the biometric determination device 130, the abandonment determination device 140, or the alarm control device 150 may be provided in a server (not shown). In addition, some or all of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, or the vehicle exterior determination unit 135 may be provided on the server, or some or all of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, or the abandonment determination unit 14 may be provided on the server, or some or all of the abandonment determination result acquisition unit 151 or the alarm control unit 152 may be provided on the server.

[0151] 16A and 16B are diagrams illustrating an example of a hardware configuration of abandonment determination device 140 according to the first embodiment. In the first embodiment, the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandonment determination unit 14 are realized by the processing circuit 1001. That is, the abandonment determination device 140 includes the processing circuit 1001 for performing control to determine whether or not an infant has been left behind in the vehicle interior. The processing circuit 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 16B.

[0152] When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0153] When the processing circuit is the processor 1004, the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005 to execute the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14. That is, the abandoned vehicle determination device 140 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST4-1, ST4-2, and ST4-3 to ST6 of FIG. 13 described above. It can also be said that the programs stored in the memory 1005 cause the computer to execute the procedures or methods of the processing of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14. Here, the memory 1005 corresponds to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0154] It is also possible to realize some of the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, the communication status information acquisition unit 143, and the abandoned vehicle determination unit 14 with dedicated hardware and some with software or firmware. For example, the functions of the vehicle interior determination result acquisition unit 141, the vehicle exterior determination result acquisition unit 142, and the communication status information acquisition unit 143 can be realized by the processing circuit 1001 as dedicated hardware, and the function of the abandoned vehicle determination unit 14 can be realized by the processor 1004 reading and executing a program stored in the memory 1005. The abandoned vehicle determination device 140 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the radio wave sensor 11, the alarm control device 150, or the vehicle information device 40.

[0155] The hardware configuration of the living body determination device 130 according to the first embodiment is the same as the hardware configuration of the abandonment determination device 140 described using FIGS. 16A and 16B, and therefore is not shown in the drawings. The functions of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135 are realized by the processing circuit 1001. That is, the biometric determination device 130 includes the processing circuit 1001 for performing vehicle interior determination processing and vehicle exterior determination processing based on the three-dimensional image generated by the radio wave sensor 11 and the biometric information extracted by the radio wave sensor 11. The processing circuit 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 that executes a program stored in a memory 1005 as shown in FIG. 16B.

[0156] The processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135. That is, the biometric determination device 130 includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of step ST1-1 and steps ST1-2 to ST3 in FIG. 13 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the biometric information acquisition unit 1301, the three-dimensional image acquisition unit 1302, the vehicle interior determination unit 131, and the vehicle exterior determination unit 135. The biometric determination device 130 includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as an abandonment determination device 140.

[0157] The hardware configuration of the alarm control device 150 according to the first embodiment is the same as the hardware configuration of the abandonment determination device 140 described using FIGS. 16A and 16B, and therefore is not shown in the drawings. The functions of the abandonment determination result acquisition unit 151 and the alarm control unit 152 are realized by the processing circuit 1001. That is, the alarm control device 150 includes the processing circuit 1001 for controlling the output of an alarm instruction based on the abandonment determination result by the abandonment determination device 140. The processing circuit 1001 may be dedicated hardware as shown in FIG. 16A, or may be a processor 1004 that executes a program stored in a memory 1005 as shown in FIG. 16B.

[0158] The processing circuit 1001 reads out and executes a program stored in the memory 1005, thereby performing the functions of the abandonment determination result acquisition unit 151 and the alarm control unit 152. That is, the alarm control device 150 includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST101 to ST102 in Fig. 15 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the abandonment determination result acquisition unit 151 and the alarm control unit 152. The alarm control device 150 includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the abandoned object determination device 140 or the alarm device 160.

[0159] As described above, according to the first embodiment, the abandoned vehicle determination system 100 includes a radio wave sensor 11 that is provided in a vehicle interior and detects a moving object based on a radio wave radiated toward the vehicle interior and reflected by an object in the vehicle interior, a biometric information acquisition unit 1301 that acquires biometric information extracted based on a detection result of the moving object detected by the radio wave sensor 11, a three-dimensional image acquisition unit 1302 that acquires a three-dimensional image that represents in three dimensions the distribution of an area where the moving object exists in the vehicle interior, which is generated based on the detection result of the moving object detected by the radio wave sensor 11, and a three-dimensional image acquisition unit 1302 that acquires a three-dimensional image that represents in three dimensions the distribution of an area where the moving object exists in the vehicle interior. The vehicle interior determination unit 1301 performs a vehicle interior determination as to whether or not there is a moving object that is deemed to be a person in the vehicle interior based on the biometric information acquired by the vehicle interior determination unit 1301; an outside vehicle interior determination unit 135 performs a vehicle interior determination as to whether or not there is a moving object that is deemed to be a person outside the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit 1302; and an abandonment determination unit 14 determines whether or not a person in the vehicle interior has been left behind based on the result of the vehicle interior determination performed by the vehicle interior determination unit 131 (vehicle interior determination result) and the result of the vehicle exterior determination performed by the vehicle exterior determination unit 135 (vehicle exterior determination result). Therefore, the abandonment determination system 100 can prevent erroneous determination of whether or not a person requiring care, such as an infant, has been left behind in the vehicle.

[0160] In addition, in the abandonment determination system 100, the abandonment determination unit 14 is provided with a first determination unit that, when the first vehicle interior determination unit 132 determines in the first vehicle interior determination that there is no moving object deemed to be a person requiring assistance (infant) inside the vehicle interior and the second vehicle interior determination unit 133 determines in the second vehicle interior determination that there is a moving object deemed to be a person requiring assistance (infant) inside the vehicle interior, determines that abandonment has occurred if the vehicle exterior determination unit 135 determines in the vehicle exterior determination that there is no moving object deemed to be a person outside the vehicle interior, and suspends determination as to whether abandonment has occurred if the vehicle exterior determination unit 135 determines in the vehicle exterior determination that there is a moving object deemed to be a person outside the vehicle interior. Therefore, the abandonment determination system 100 can suppress erroneous determination of whether or not a person requiring assistance has been left behind in the vehicle compartment.

[0161] In addition, in the abandonment determination system 100, the abandonment determination unit 14 is provided with a second determination unit 145 that makes a provisional determination (first abandonment determination) based on the determination by the first determination unit 144 of whether abandonment has occurred or not (second abandonment determination) and that makes a final determination (second abandonment determination) of whether abandonment has occurred or not based on the result of the provisional determination and communication status information regarding the communication status with the short-range wireless communication device 45 outside the vehicle cabin obtained from a vehicle information device 40 that is mounted inside the vehicle cabin and monitors the communication status with the short-range wireless communication device 45 outside the vehicle cabin. The abandonment determination system 100 determines whether a person requiring assistance has been left behind in the vehicle cabin by combining the 3D image and biometric information output from the radio wave sensor 11 with communication status information, thereby confirming whether a person requiring assistance has been left behind in the vehicle cabin and suppressing erroneous determinations that a person requiring assistance has been left behind in the vehicle cabin.

[0162] More specifically, in the abandonment determination system 100, when the first vehicle interior determination unit 132 determines in the first vehicle interior determination that there is no moving object that is regarded as a person requiring care (infant) inside the vehicle interior and the second vehicle interior determination unit 133 determines in the second vehicle interior determination that there is a moving object that is regarded as a person requiring care inside the vehicle interior, and when the vehicle exterior determination unit 135 determines in the vehicle exterior determination that there is no moving object that is regarded as a person outside the vehicle interior, the first determination unit 144 provisionally determines that abandonment has occurred, and when the vehicle exterior determination unit 135 determines in the vehicle exterior determination that there is no moving object that is regarded as a person outside the vehicle interior, If the first judgment unit 144 determines that there is a moving object that is regarded as a person outside the vehicle cabin, the second judgment unit 145 suspends the judgment as to whether or not abandonment has occurred, and if the first judgment unit 144 provisionally judges that abandonment has occurred or suspends the judgment as to whether or not abandonment has occurred, the second judgment unit 145 determines that abandonment has not occurred if communication between the vehicle and the short-range wireless communication device 45 is established based on the communication status information, and determines that abandonment has occurred if communication between the vehicle and the short-range wireless communication device is not established. The abandonment determination system 100 determines whether a person requiring assistance has been left behind in the vehicle cabin by combining the 3D image and biometric information output from the radio wave sensor 11 with communication status information, thereby confirming whether a person requiring assistance has been left behind in the vehicle cabin and suppressing erroneous determinations that a person requiring assistance has been left behind in the vehicle cabin.

[0163] In addition, the abandonment determination system 100 includes an alarm control unit 152 that outputs an alarm instruction to output an alarm when the first determination unit 144 determines that abandonment has occurred, and the alarm control unit 152 can be configured not to output an alarm instruction when the first determination unit 144 suspends determination of whether abandonment has occurred. Therefore, the abandonment determination system 100 can suppress false alarms when, for example, it is erroneously determined that a person requiring assistance has been left behind in the vehicle compartment even though the person requiring assistance has not been left behind.

[0164] Furthermore, according to embodiment 1, the abandonment determination device 140 is configured to include: an interior vehicle interior determination result acquisition unit 141 that acquires an interior vehicle interior determination result of whether or not a moving object that is deemed to be a person requiring assistance (infant) is present inside the vehicle interior, the interior vehicle interior determination result acquisition unit 141 being configured to acquire an interior vehicle interior determination result of whether or not a moving object that is deemed to be a person requiring assistance is present outside the vehicle interior, the interior vehicle interior determination result acquisition unit 142 being configured to acquire an exterior vehicle interior determination result of whether or not a moving object that is deemed to be a person is present outside the vehicle interior, the exterior vehicle interior determination result acquisition unit 142 being configured to acquire an exterior vehicle interior determination result of whether or not a moving object that is deemed to be a person is present outside the vehicle interior, the exterior vehicle interior determination result acquisition unit 14 being configured to determine whether or not a person requiring assistance has been left behind inside the vehicle interior, based on the interior vehicle interior determination result acquired by the interior vehicle interior determination result acquisition unit 141 and the exterior vehicle interior determination result acquired by the exterior vehicle interior determination result acquisition unit 142. Therefore, the abandonment determination device 140 can prevent erroneous determination of whether or not a person requiring care, such as an infant, has been left behind in the vehicle.

[0165] Any of the components of the embodiments may be modified or omitted. [Industrial Applicability]

[0166] The abandonment determination system according to the present disclosure can prevent erroneous determination of whether or not a child or the like has been left behind in the vehicle. [Explanation of symbols]

[0167] 100 Abandonment determination system, 11 radio wave sensor, 110 radio wave transmitting and receiving unit, 111 transmitting antenna, 112 receiving antenna, 113 high frequency signal generating circuit unit, 114 transmitting circuit unit, 115 receiving circuit unit, 116 A / D conversion unit, 120 sensor signal processing unit, 121 3D image generating unit, 122 biometric information extraction unit, 130 biometric determination device, 1301 biometric information acquisition unit, 1302 3D image acquisition unit, 131 vehicle interior determination unit, 132 first vehicle interior determination unit, 133 second vehicle interior determination unit, 135 vehicle exterior determination unit, 140 Abandonment determination device, 141 vehicle interior determination result acquisition unit, 142 vehicle exterior determination result acquisition unit, 143 communication status information acquisition unit, 14 abandonment determination unit, 144 first determination unit, 145 second determination unit, 150 alarm control device, 151 Abandonment determination result acquisition unit, 152 alarm control unit, 160 alarm device, 40 vehicle information device, 41 wireless communication status monitoring unit, 45 short-range wireless communication device, 1001 processing circuit, 1002 input interface device, 1003 output interface device, 1004 processor, 1005 memory.

Claims

1. a radio wave sensor provided in a vehicle interior, configured to detect a moving object based on a radio wave reflected by an object present within a range to which the radio wave can be irradiated, including inside and outside the vehicle interior, said radio wave sensor detecting the moving object based on a detection result of the moving object detected by the radio wave sensor; and a biometric information acquiring unit acquiring biometric information extracted based on a detection result of the moving object detected by the radio wave sensor. a three-dimensional image acquisition unit that acquires a three-dimensional image that represents in three dimensions a distribution of an area where the moving object exists within a range that can be irradiated with the radio waves, including inside and outside the vehicle cabin, generated based on the detection result of the moving object detected by the radio wave sensor; and a vehicle interior determination unit that determines whether or not a moving object deemed to be a person requiring assistance is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit and the biometric information acquired by the biometric information acquisition unit; an outside-vehicle determination unit that determines whether the moving object regarded as a person is present outside the vehicle cabin by determining whether the moving object regarded as a person is present in a portion of the three-dimensional image corresponding to an area outside the vehicle cabin based on the three-dimensional image acquired by the three-dimensional image acquisition unit; and an abandonment determination unit that determines whether the person requiring assistance has been abandoned inside the vehicle cabin based on a determination result of the vehicle interior determination performed by the vehicle interior determination unit and a determination result of the vehicle exterior determination performed by the vehicle exterior determination unit; A system for determining whether a child is left behind.

2. The vehicle interior determination unit a first vehicle interior determination unit that performs a first vehicle interior determination as to whether or not the moving object regarded as the person requiring assistance is present in the vehicle interior based on the three-dimensional image acquired by the three-dimensional image acquisition unit; a second vehicle interior determination unit that performs a second vehicle interior determination as to whether the moving body regarded as the person requiring assistance is present in the vehicle interior based on the biometric information acquired by the biometric information acquisition unit.

2. The abandonment determination system according to claim 1,

3. The abandonment determination unit a first determination unit that, when the first vehicle interior determination unit determines in the first vehicle interior determination that the moving object regarded as the person requiring assistance is not present inside the vehicle interior and the second vehicle interior determination unit determines in the second vehicle interior determination that the moving object regarded as the person requiring assistance is present inside the vehicle interior, determines that the abandonment has occurred if the vehicle exterior determination unit determines in the vehicle exterior determination that the moving object regarded as the person is not present outside the vehicle interior, and suspends determination of whether the abandonment has occurred if the vehicle exterior determination unit determines in the vehicle exterior determination that the moving object regarded as the person is present outside the vehicle interior.

3. The abandonment determination system according to claim 2.

4. The first determination unit If the state in which the determination as to whether or not the vehicle has been left behind continues for a first suspension determination time or more, it is determined that the vehicle has been left behind.

4. The abandonment determination system according to claim 3.

5. The abandonment determination unit The vehicle information system further includes a second determination unit that determines whether the vehicle has been abandoned based on a provisional determination made by the first determination unit and communication status information regarding a communication status with the short-range wireless communication device outside the vehicle cabin, the communication status information being acquired from a vehicle information device that is mounted inside the vehicle cabin and monitors a communication status with the short-range wireless communication device outside the vehicle cabin.

4. The abandonment determination system according to claim 3.

6. The first determination unit When the first vehicle interior determination unit determines in the first vehicle interior determination that the moving object regarded as the person requiring assistance is not present inside the vehicle interior, and the second vehicle interior determination unit determines in the second vehicle interior determination that the moving object regarded as the person requiring assistance is present inside the vehicle interior, if the vehicle exterior determination unit determines in the vehicle exterior determination that the moving object regarded as the person is not present outside the vehicle interior, it provisionally determines that the abandonment has occurred, and if the vehicle exterior determination unit determines in the vehicle exterior determination that the moving object regarded as the person is present outside the vehicle interior, it suspends the determination of whether the abandonment has occurred, When the first determination unit provisionally determines that the abandonment has occurred or when the determination as to whether the abandonment has occurred is suspended, the second determination unit determines, based on the communication state information, that the abandonment has not occurred if communication between the vehicle and the short-range wireless communication device is established, and determines that the abandonment has occurred if communication between the vehicle and the short-range wireless communication device is not established.

6. The abandonment determination system according to claim 5.

7. an alarm control unit that outputs an alarm instruction to output an alarm when the first determination unit determines that the abandonment has occurred; The warning control unit does not output the warning instruction when the first determination unit suspends the determination of whether the abandonment has occurred.

4. The abandonment determination system according to claim 3.

8. The alarm control unit When the state in which the first determination unit has suspended the determination of whether or not the vehicle has been left behind continues for a second suspension determination time or more, the first determination unit outputs the warning instruction.

8. The abandonment determination system according to claim 7.

9. The second determination unit includes an alarm control unit that outputs an alarm instruction to output an alarm when it determines that the vehicle has been left behind.

7. The abandonment determination system according to claim 5 or 6.

10. an alarm control unit that outputs an alarm instruction to output an alarm when the first determination unit provisionally determines that the abandonment has occurred or when the second determination unit determines that the abandonment has occurred; The warning control unit selects whether to output the warning instruction based on a result of the provisional determination by the first determination unit or to output the warning instruction based on a result of the main determination by the second determination unit.

7. The abandonment determination system according to claim 5 or 6.

11. an interior determination result acquisition unit that acquires an interior determination result of whether or not a moving object that is deemed to be a person requiring care is present in the vehicle interior, the determination result being based on a three-dimensional image that represents in three dimensions the distribution of areas where the moving object exists in an area where the radio waves can be irradiated, including inside and outside the vehicle interior, generated based on a detection result of the moving object detected by a radio wave sensor that is provided in the vehicle interior and that detects a moving object based on a radio wave that is radiated toward the vehicle interior and that is reflected by an object that exists in an area where the radio waves can be irradiated, including inside and outside the vehicle interior, and biological information extracted based on the detection result of the moving object detected by the radio wave sensor; an outside-cabin determination result acquisition unit that acquires an outside-cabin determination result of whether or not the moving object regarded as a person is present outside the vehicle cabin, the outside-cabin determination result being obtained by determining whether or not the moving object regarded as a person is present in a portion of the three-dimensional image corresponding to an area outside the vehicle cabin, based on the three-dimensional image; an abandonment determination unit that determines whether the person requiring assistance has been abandoned inside the vehicle compartment based on the vehicle interior determination result acquired by the vehicle interior determination result acquisition unit and the vehicle exterior determination result acquired by the vehicle exterior determination result acquisition unit; An abandonment determination device comprising:

12. The vehicle interior determination result is a first vehicle interior determination result, based on the three-dimensional image, as to whether or not the moving object regarded as the person requiring assistance is present in the vehicle interior; and a second vehicle interior determination result, based on the biological information, as to whether or not the moving object regarded as the person requiring assistance is present in the vehicle interior. The abandonment determination unit a first determination unit that, when the first vehicle interior determination result indicates that the moving object regarded as the person requiring assistance is not present inside the vehicle interior and the second vehicle interior determination result indicates that the moving object regarded as the person requiring assistance is present inside the vehicle interior, determines that the person has been abandoned if the vehicle exterior determination result indicates that the moving object regarded as the person is not present outside the vehicle interior, and suspends determination of whether the person has been abandoned if the vehicle exterior determination result indicates that the moving object regarded as the person is present outside the vehicle interior.

12. The abandonment determination device according to claim 11.

13. The first determination unit If the state in which the determination as to whether or not the vehicle has been left behind continues for a first suspension determination time or more, it is determined that the vehicle has been left behind.

13. The abandonment determination device according to claim 12.

14. The abandonment determination unit The vehicle information system further includes a second determination unit that determines whether the vehicle has been abandoned based on a provisional determination made by the first determination unit and communication status information regarding a communication status with the short-range wireless communication device outside the vehicle cabin, the communication status information being acquired from a vehicle information device that is mounted inside the vehicle cabin and monitors a communication status with the short-range wireless communication device outside the vehicle cabin.

14. The abandonment determination device according to claim 13.

15. The first determination unit When the first vehicle interior determination result indicates that the moving object regarded as the person requiring assistance is not present within the vehicle interior and the second vehicle interior determination result indicates that the moving object regarded as the person requiring assistance is present within the vehicle interior, if the vehicle exterior determination result indicates that the moving object regarded as the person is not present outside the vehicle interior, it is provisionally determined that the person has been abandoned, and if the vehicle exterior determination result indicates that the moving object regarded as the person is present outside the vehicle interior, the determination of whether the person has been abandoned is suspended; When the first determination unit provisionally determines that the abandonment has occurred or when the determination as to whether the abandonment has occurred is suspended, the second determination unit determines, based on the communication state information, that the abandonment has not occurred if communication between the vehicle and the short-range wireless communication device is established, and determines that the abandonment has occurred if communication between the vehicle and the short-range wireless communication device is not established.

15. The abandonment determination device according to claim 14.

16. an alarm control unit that outputs an alarm instruction to output an alarm when the first determination unit determines that the abandonment has occurred; The warning control unit does not output the warning instruction when the first determination unit suspends the determination of whether the abandonment has occurred.

13. The abandonment determination device according to claim 12.

17. The alarm control unit When the state in which the first determination unit has suspended the determination of whether or not the vehicle has been left behind continues for a second suspension determination time or more, the first determination unit outputs the warning instruction.

17. The abandonment determination device according to claim 16.

18. The second determination unit includes an alarm control unit that outputs an alarm instruction to output an alarm when it determines that the vehicle has been left behind.

16. The abandonment determination device according to claim 14 or 15.

19. an alarm control unit that outputs an alarm instruction to output an alarm when the first determination unit provisionally determines that the abandonment has occurred or when the second determination unit determines that the abandonment has occurred; The warning control unit selects whether to output the warning instruction based on a result of the provisional determination by the first determination unit or to output the warning instruction based on a result of the main determination by the second determination unit.

16. The abandonment determination device according to claim 14 or 15.

20. a step of acquiring a vehicle interior determination result of whether or not a moving object deemed to be a person requiring assistance is present in the vehicle interior, the vehicle interior determination result acquisition unit being provided in the vehicle interior and configured to detect a moving object based on a radio wave sensor that detects a moving object based on a radio wave radiated toward the vehicle interior and reflected by an object present in a range where the radio wave can be irradiated, including inside and outside the vehicle interior, and a three-dimensional image that represents in three dimensions the distribution of areas where the moving object exists in a range where the radio wave can be irradiated, including inside and outside the vehicle interior, generated based on the detection result of the moving object detected by the radio wave sensor, and based on biological information extracted based on the detection result of the moving object detected by the radio wave sensor; a step of acquiring an outside-of-vehicle-compartment determination result indicating whether or not the moving object regarded as a person is present outside the vehicle compartment, the outside-of-vehicle-compartment determination result being acquired by determining whether or not the moving object regarded as a person is present in a portion of the three-dimensional image corresponding to an area outside the vehicle compartment; a step in which an abandonment determination unit determines whether the person requiring assistance has been left inside the vehicle compartment based on the vehicle interior determination result acquired by the vehicle interior determination result acquisition unit and the vehicle exterior determination result acquired by the vehicle exterior determination result acquisition unit; A method for determining whether an object is left behind, comprising:

Images