Information processing device, detection sensor, information processing system, and information processing method

Abstract

This invention comprises: an image acquisition unit (102) that acquires, from a detection sensor (2, 2a) which is provided inside a vehicle and includes a camera (22) and a radio wave sensor (21), a captured image captured by the camera (22); a camera error detection unit (103) that detects a camera (22) installation error, on the basis of the captured image; a radio wave sensor error calculation unit (104) that calculates a radio wave sensor (21) installation error, on the basis of the camera (22) installation error detected by the camera error detection unit (103); and an output unit (106, 106a, 106b) that outputs error consideration information which is for converting moving body distribution information generated by the radio wave sensor (21) into moving body distribution information which takes into account the radio wave sensor (21) installation error calculated by the radio wave sensor error calculation unit (104).

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a detection sensor, an information processing system, and an information processing method. [Background technology]

[0002] Conventionally, a technology for monitoring vehicle occupants based on information generated from reflected waves received by a radio wave sensor that is installed inside the vehicle cabin, emits radio waves into the vehicle cabin, and receives reflected waves from objects to detect the position and movement of moving objects inside the vehicle cabin has been known. Incidentally, a known technique for correcting misalignment in the installation position or orientation of a camera installed inside a vehicle cabin involves using a camera to capture an image of a marker or the like whose direction or distance from the camera is known, and adjusting the installation position or orientation of the camera or correcting the captured image based on the amount of misalignment between the position of the marker or the like in the captured image and the position of the marker or the like that should have been captured in the captured image (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-006175 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, when a radio wave sensor is installed inside a vehicle, it is fixed inside the vehicle via some kind of structure, such as the ceiling or dashboard, between it and the vehicle frame. Therefore, when a radio wave sensor is installed inside a vehicle, there is a possibility that the installation position of the radio wave sensor or the direction of the radio waves emitted by the radio wave sensor may be misaligned. Hereinafter, a misalignment in the installation position of the radio wave sensor or the direction of the radio waves emitted by the radio wave sensor will be referred to as an "installation error of the radio wave sensor." As a result, there is a problem that the information output from the radio wave sensor may have been generated when an installation error occurred in the radio wave sensor. If vehicle occupants are monitored based on information generated when an installation error occurred in the radio wave sensor, the accuracy of the monitoring results may be reduced. Note that the technology typified by the technology disclosed in Patent Document 1 is a technology for adjusting the installation position or orientation of the camera or correcting the captured image, but does not take into account installation errors of the radio wave sensor, and therefore still cannot solve the above problem.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an information processing device that can provide information (hereinafter referred to as "error-taking information") that can be used as information obtained without installation error of the radio wave sensor, generated based on the reflected waves received by a radio wave sensor that is installed in the vehicle cabin, emits radio waves into the vehicle cabin, and can detect the position and movement of moving objects in the vehicle cabin by receiving reflected waves of the emitted radio waves reflected by an object. [Means for solving the problem]

[0006] The information processing device according to the present disclosure includes an image acquisition unit that acquires an image captured by the camera from a detection sensor having a camera installed within the vehicle cabin that captures an image of a target area within the vehicle cabin where at least an occupant may be present, and a radio wave sensor that generates moving object distribution information that three-dimensionally represents the distribution of areas within the vehicle cabin where moving objects are present, based on radio waves emitted toward the target area within the vehicle cabin and reflected by objects within the vehicle cabin; a camera error detection unit that detects camera installation errors based on the image acquired by the image acquisition unit; a radio wave sensor error calculation unit that calculates radio wave sensor installation errors based on the camera installation errors detected by the camera error detection unit; and an output unit that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor into moving object distribution information that takes into account the radio wave sensor installation errors calculated by the radio wave sensor error calculation unit. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide error-taking information that allows information generated based on the reflected waves received by a radio wave sensor that is installed within the vehicle cabin, emits radio waves into the vehicle cabin, and is capable of detecting the position and movement of moving objects within the vehicle cabin by receiving reflected waves of the emitted radio waves reflected by an object, to be used as information obtained without any installation error of the radio wave sensor. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing device according to a first embodiment. [Figure 2] 2 is a diagram for explaining a configuration example of a detection sensor according to the first embodiment. FIG. [Figure 3] 10A and 10B are diagrams for explaining other configuration examples of the detection sensor according to the first embodiment. [Figure 4] 2 is a configuration diagram showing a radio wave sensor mounted on the detection sensor in the first embodiment. FIG. [Figure 5] 2 is a block diagram showing the functions of a radio wave sensor mounted on a vehicle in the first embodiment. FIG. [Figure 6] FIG. 2 is an explanatory diagram showing an example of an FM signal Tx(k) generated by a high-frequency signal generating circuit. [Figure 7] FIG. 2 is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave. [Figure 8] 10 is an explanatory diagram showing the relationship between the sweep time T of an FM transmission wave and the frequency difference fd. FIG. [Figure 9] FIG. 2 is an explanatory diagram showing an example of a virtual antenna configured in a radio wave sensor. [Figure 10] 10A, 10B, and 10C are diagrams showing examples of a moving object map generated by the radio wave sensor in the first embodiment. [Figure 11] 4 is a flowchart illustrating the operation of the information processing device according to the first embodiment. [Figure 12]Figures 12A and 12B are diagrams illustrating an example of the effect that would occur on a monitoring device that monitors occupants using a moving object map generated by a radio wave sensor if there were a discrepancy in the position and angle of a moving object detected by the radio wave sensor. [Figure 13] 13A and 13B are diagrams illustrating an example of a hardware configuration of an information processing device according to the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of the configuration of an information processing device according to a second embodiment. [Figure 15] 10 is a flowchart illustrating the operation of the information processing device according to the second embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of the configuration of an information processing device according to a third embodiment. [Figure 17] 11 is a flowchart illustrating the operation of the information processing device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 The information processing device according to the first embodiment provides information (hereinafter referred to as "error-considered information") that allows a monitoring device that monitors occupants in the vehicle cabin to use the moving object distribution information as information obtained without installation errors of the radio wave sensor, based on information (hereinafter referred to as "moving object distribution information") that is generated by a radio wave sensor installed in the vehicle cabin and that represents the distribution of areas where moving objects exist in the vehicle cabin in three dimensions. The radio wave sensor detects moving objects based on waves reflected by objects in the vehicle cabin when radio waves are emitted toward an area in the vehicle cabin where at least occupants may be present (hereinafter referred to as "target area"), and generates moving object distribution information. In the following description, "radio wave sensor installation error" refers to a deviation in the installation position of the radio wave sensor in the real space inside the vehicle cabin (installation position error), or a deviation in the direction of the radio waves emitted by the radio wave sensor in the real space inside the vehicle cabin (radio wave direction error). More specifically, "radio wave sensor installation error" refers to an error in the installation position of the radio wave sensor relative to a predetermined installation position, or an error in the direction of the radio waves emitted by the radio wave sensor relative to a predetermined direction. Note that the "error" referred to here does not include errors that are practically negligible, but refers to errors that affect a monitoring device that performs processing using moving object distribution information generated by the radio wave sensor. In the following description, when the term "direction of the radio wave sensor" is used, the term "direction of the radio wave sensor" means "the direction of the radio waves emitted by the radio wave sensor."

[0011] In the first embodiment, the error-considered information is, in detail, moving object distribution information generated by the radio wave sensor after correcting it so that it becomes moving object distribution information obtained without any installation error of the radio wave sensor (hereinafter referred to as "corrected moving object distribution information"). The information processing device according to the first embodiment generates the corrected moving object distribution information and outputs it to the monitoring device. In the first embodiment, the moving object distribution information is what is called a "moving object map." In the following description, the moving object distribution information is also referred to as a moving object map, and the corrected moving object distribution information is also referred to as a corrected moving object map. Details of the moving object map will be described later.

[0012] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device 1 according to the first embodiment. The information processing device 1 is connected to a detection sensor 2 and a monitoring device 3. The information processing device 1 and the detection sensor 2 constitute an information processing system 4. The detection sensor 2 is equipped with a radio wave sensor 21 and a camera 22 . The information processing device 1 corrects the moving object map based on the captured image captured by the camera 22 mounted on the detection sensor 2 and the moving object map generated by the radio wave sensor 21 mounted on the detection sensor 2, and outputs the corrected moving object map to the monitoring device 3 as error-taking information. The monitoring device 3 includes, for example, an occupant detection device 31 and a physique determination device 32, and monitors the occupants in the vehicle cabin based on the corrected moving object map output from the information processing device 1. In the first embodiment, the information processing device 1, the detection sensor 2, and the monitoring device 3 are mounted on, for example, a vehicle (not shown). The information processing device 1, the occupant detection device 31, and the physique determination device 32 will be described in detail later, and first, the detection sensor 2 according to the first embodiment will be described in detail.

[0013] The detection sensor 2 is provided in the vehicle interior, for example, on an overhead console. Note that this is merely an example, and the detection sensor 2 may be provided in a location other than the overhead console, such as on the ceiling or dashboard. The detection sensor 2 is provided in a location in the vehicle interior where the camera 22 can capture an image of the target area and the radio wave sensor 21 can detect a moving object present in the target area; in other words, where the radio wave sensor 21 and the camera 22 can view the target area in the vehicle interior. Note that the target area captured by the camera 22 and the target area to which the radio wave sensor 21 emits radio waves do not have to coincide.

[0014] The detection sensor 2 has a radio wave sensor 21 that emits radio waves in a predetermined direction (hereinafter referred to as "reference sensor direction") at a predetermined installation position (hereinafter referred to as "reference sensor position") that is set in advance in the vehicle interior, and a camera 22 that emits radio waves in a predetermined installation position (hereinafter referred to as "camera reference position") that is set in advance. in,The radio wave sensor 21 is installed so as to face a predetermined direction (hereinafter referred to as the "reference camera direction"). However, as described above, the detection sensor 2 is fixed inside the vehicle cabin via some structure, such as the dashboard, between it and the vehicle body frame, and therefore installation error of the radio wave sensor 21 installed inside the vehicle cabin may occur.

[0015] In the first embodiment, the position of radio wave sensor 21 is represented by the center position of radio wave sensor 21. The position of camera 22 is represented by the center position of camera 22. The positions of radio wave sensor 21 and camera 22 are represented in a three-dimensional coordinate system that represents the real space inside the vehicle cabin. In the three-dimensional coordinate system representing the real space inside the vehicle cabin, the x-axis is an axis parallel to the vehicle width direction, the y-axis is an axis parallel to the vehicle height direction, and the z-axis is an axis parallel to the vehicle length direction. Note that in the first embodiment, "parallel" is not limited to being strictly "parallel" and includes "approximately parallel."

[0016] 2 and 3 are diagrams for explaining a configuration example of the detection sensor 2 according to the first embodiment. 2 and 3, the upward direction in the drawings corresponds to the upward direction when the detection sensor 2 is installed inside the vehicle cabin. In FIG. 2, for the sake of convenience, only the radio wave sensor 21, the camera 22, and the fixed part 23 are shown as components of the detection sensor 2. 3, for the sake of convenience, only the camera 22, the transmitting antenna 11, the receiving antenna 12, and the substrate 24 are shown as components of the detection sensor 2.

[0017] 2 , the detection sensor 2 includes a fixed part 23, a camera 22 provided on the fixed part 23, and a radio wave sensor 21 provided on the fixed part 23. When the detection sensor 2 is installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 and the three-dimensional coordinate axes of the radio wave sensor 21 are configured to coincide. The three-dimensional coordinate axes of the camera 22 are the x-axis, y-axis, and z-axis specific to the camera 22, and the three-dimensional coordinate axes of the radio wave sensor 21 are the x-axis, y-axis, and z-axis specific to the radio wave sensor 21. In the first embodiment, the x-axis and y-axis specific to the camera 22 refer to axes on the plane of the imaging element of the camera 22, and the z-axis refers to an axis normal to the imaging element. In the first embodiment, the x-axis and y-axis specific to the radio wave sensor 21 refer to axes on the surface of the substrate 24 on which the antennas (transmitting antenna 11 and receiving antenna 12) are arranged, and the z-axis refers to an axis normal to the surface of the substrate 24. In the first embodiment, "match" is not limited to a perfect match, but also includes a match within an allowable error range.

[0018] As shown in Figure 3, the detection sensor 2 may, for example, include a camera 22, a radio wave sensor 21, and a substrate 24 on which the camera, the transmitting antenna 11 (see Figure 4 described below) of the radio wave sensor 21, and the receiving antenna 12 (see Figure 4 described below) of the radio wave sensor 21 are arranged on a common surface, and the camera 22 may be configured to be arranged between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24. The transmitting antenna 11 shown in FIG. 3 is, more specifically, transmitting antenna elements 11-1 and 11-2 included in the transmitting antenna 11 (see FIG. 4 described later), and the receiving antenna 12 shown in FIG. 3 is, more specifically, receiving antenna elements 12-1 to 12-4 included in the receiving antenna 12 (see FIG. 4 described later). Although only one transmitting antenna element 11-1 or 11-2 is shown in FIG. 3, the transmitting antenna 11 may have a plurality of transmitting antenna elements 11-1 or 11-2 in the radio wave sensor 21. Furthermore, although three receiving antenna elements 12-1 to 12-4 are shown in FIG. 3, the number of receiving antenna elements 12-1 to 12-4 included in the receiving antenna 12 in the radio wave sensor 21 is not limited to three. The receiving antenna 12 has a plurality of receiving antenna elements 12-1 to 12-4. The radio wave sensor 21 will be described in detail later.

[0019] The camera 22 captures an image of a target area within the vehicle cabin. Camera 22 is, for example, a near-infrared camera or a visible light camera. In the first embodiment, camera 22 is assumed to be, for example, a camera shared with a so-called DMS (Driver Monitoring System) that is installed for the purpose of monitoring the interior of a vehicle.

[0020] The radio wave sensor 21 generates moving object distribution information, i.e., a moving object map, that represents the distribution of areas where moving objects exist within the vehicle cabin in three dimensions based on the reflected waves of radio waves emitted toward a target area within the vehicle cabin and reflected by objects within the vehicle cabin. The radio wave sensor 21 is, for example, a millimeter wave radar.

[0021] The detection sensor 2 outputs the captured image captured by the camera 22 and the moving object map generated by the radio wave sensor 21 to the information processing device 1.

[0022] Here, the radio wave sensor 21 according to the first embodiment will be described in detail.

[0023] FIG. 4 is a configuration diagram showing the radio wave sensor 21 mounted on the detection sensor 2 in the first embodiment. FIG. 5 is a block diagram showing the functions of the radio wave sensor 21 mounted on the vehicle in the first embodiment. Of the three-dimensional coordinate axes in FIG. 4, the x-axis is an axis parallel to the vehicle width direction, the y-axis is an axis parallel to the vehicle height direction, and the z-axis is an axis parallel to the vehicle length direction.

[0024] The transmitting antenna 11 is a planar antenna configured on an electronic circuit board. The transmitting antenna 11 has a plurality of transmitting antenna elements 11-1 and 11-2 that radiate radio waves toward a target area.

[0025] The installation positions of the transmitting antenna element 11-1 and the transmitting antenna element 11-2 are different from each other in the vehicle height direction. In the radio wave sensor 21 shown in FIG. 4, the installation position of the transmitting antenna element 11-1 in the vehicle height direction is higher than the installation position of the transmitting antenna element 11-2 in the vehicle height direction.

[0026] The receiving antenna 12 is a planar antenna configured on an electronic circuit board, and is installed on the same plane as the transmitting antenna 11. However, the same plane here does not mean that the plane on which the transmitting antenna 11 and the plane on which the receiving antenna 12 are installed are strictly the same, and also includes different planes as long as there are no practical problems. The receiving antenna 12 has a plurality of receiving antenna elements 12-1 to 12-4 that receive the reflected waves of the radio waves radiated from the transmitting antenna 11.

[0027] The receiving antenna element 12-1, the receiving antenna element 12-2, the receiving antenna element 12-3, and the receiving antenna element 12-4 are installed at different positions in the width direction of the vehicle.

[0028] The radio wave sensor circuit section 13 includes a high-frequency signal generating circuit 14, a radio wave transmitting section 15, a radio wave receiving section 16, an analog-to-digital conversion circuit (hereinafter referred to as the “A / D conversion circuit”) 17, a map generating section 18, a communication circuit 19, and a power supply circuit 200. The high frequency signal generating circuit 14, the radio wave transmitting section 15, and the radio wave receiving section 16 constitute a radio wave transmitting / receiving section 130 that radiates radio waves from the transmitting antenna 11 and acquires a received signal based on the reflected waves from the receiving antenna 12.

[0029] High frequency signal generating circuit 14 generates an FM (Frequency Modulation) signal, the frequency of which changes over time, as a sensing signal, and outputs the FM signal to radio wave transmitting unit 15 and radio wave receiving unit 16, respectively. 4 uses FM-CW (Frequency Modulation-Continuous Wave) as the modulation method, and high-frequency signal generating circuit 14 generates an FM signal. However, the modulation method is not limited to FM-CW, and for example, FCM (Fast-Chirp Modulation) may also be used. When FCM is used as the modulation method, high-frequency signal generating circuit 14 generates an FCM signal and outputs the FCM signal to radio wave transmitting unit 15 and radio wave receiving unit 16, respectively.

[0030] The radio wave transmitting unit 15 has a transmitting circuit 15-1 and a transmitting circuit 15-2. The radio wave transmitting unit 15 causes one of the transmitting antenna elements 11-1 and 11-2 to emit radio waves toward the target area. That is, the radio wave transmitting unit 15 switches between the transmitting antenna elements 11-1 and 11-2 in order to select one transmitting antenna element from which to emit radio waves. When radiating radio waves from transmitting antenna element 11-1, radio wave transmitting unit 15 causes transmitting circuit 15-1 to output an FM signal to transmitting antenna element 11-1. When radiating radio waves from transmitting antenna element 11-2, radio wave transmitting unit 15 causes transmitting circuit 15-2 to output an FM signal to transmitting antenna element 11-2.

[0031] The transmitting circuit 15-1 amplifies the FM signal output from the high frequency signal generating circuit 14 and outputs the amplified FM signal to the transmitting antenna element 11-1, thereby causing the transmitting antenna element 11-1 to radiate an FM transmission wave, which is an electric wave, toward the target area. The transmitting circuit 15-2 amplifies the FM signal output from the high frequency signal generating circuit 14 and outputs the amplified FM signal to the transmitting antenna element 11-2, thereby causing the transmitting antenna element 11-2 to radiate an FM transmission wave toward the target area.

[0032] The radio wave receiving section 16 has a receiving circuit 16-1, a receiving circuit 16-2, a receiving circuit 16-3, and a receiving circuit 16-4. When the receiving antenna element 12-m receives an FM receiving wave that is a reflected wave, the receiving circuit 16-m (m=1, 2, 3, 4) acquires a receiving signal of the FM receiving wave from the receiving antenna element 12-m. The receiving circuit 16-m detects the difference between the frequency of the FM signal output from the high frequency signal generating circuit 14 and the frequency of the received signal (hereinafter referred to as the "frequency difference") f d Extract. The receiving circuit 16-m receives the frequency difference f d an intermediate frequency signal IF having m and generates an intermediate frequency signal IF m is output to the A / D conversion circuit 17.

[0033] The A / D conversion circuit 17 converts the intermediate frequency signal IF output from the receiving circuit 16-m (m=1, 2, 3, 4) into m converting the analog signal to a digital signal D m Convert to. The A / D conversion circuit 17 converts the digital signal D m is output to the map generating unit 18.

[0034] The map generating unit 18 is realized by, for example, a digital signal processing circuit. The map generating unit 18 receives from the A / D conversion circuit 17 a digital signal D relating to the FM reception wave received by the reception circuit 16-m (m=1, 2, 3, 4). m Get. The map generator 18 generates the digital signal D m A motion map is generated based on (m=1,2,3,4). The map generating unit 18 outputs the generated moving object map to the communication circuit 19.

[0035] The digital signal processing circuit may be implemented, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The digital signal processing circuit is not limited to being realized by dedicated hardware, but may also be realized by software, firmware, or a combination of software and firmware. Software or firmware is stored as a program in the memory of a computer. A computer refers to hardware that executes the program, such as a CPU (Central Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0036] Here, a specific operation of the radio wave sensor 21 to generate a moving object map and an example of the moving object map will be described. Although it is possible to use signals modulated by various modulation methods as sensing signals for the radio wave sensor 21, an example in which a signal modulated by the FM-CW method is used will be described here. The high-frequency signal generating circuit 14 generates, as a sensing signal, an FM signal Tx(k) (k=1, . . . , K) which is a chirp signal whose frequency changes over time, as shown in Fig. 6. K is an integer equal to or greater than 2. FIG. 6 is an explanatory diagram showing an example of the FM signal Tx(k) generated by the high frequency signal generating circuit 14. As shown in FIG. In the FM signal Tx(k) shown in FIG. 6, the initial state of the frequency is the lower limit frequency, and the frequency increases with the passage of time until it reaches the upper limit frequency. In FIG. 6, the sweep time T is the time it takes for the frequency of the FM signal Tx(k) to reach the upper limit frequency from the lower limit frequency, and the frequency bandwidth BW is the frequency difference between the upper limit frequency and the lower limit frequency. In the example of FIG. 6, the FM signal Tx(k) is c and is generated K times. High frequency signal generating circuit 14 outputs FM signal Tx(k) to transmitting circuits 15-1 and 15-2 of radio wave transmitting unit 15, and outputs FM signal Tx(k) to receiving circuits 16-1 to 16-4 of radio wave receiving unit 16. FIG. 6 also shows the amplitude waveform of the FM signal Tx(k).

[0037] The transmission circuit 15-1 and the transmission circuit 15-2 alternately perform the operation of outputting the FM signal Tx(k). When the transmitting circuit 15-1 receives the FM signal Tx(k) from the high frequency signal generating circuit 14 at the timing for outputting the FM signal Tx(k), it amplifies the FM signal Tx(k) and outputs the amplified FM signal Tx(k) to the transmitting antenna element 11-1. The amplified FM signal Tx(k) is output from the transmission circuit 15-1 to the transmission antenna element 11-1, and an FM transmission wave is radiated from the transmission antenna element 11-1 toward the target area. When the transmission circuit 15-1 is outputting the FM signal Tx(k), the transmission circuit 15-2 stops outputting the FM signal Tx(k). The timing at which the transmission circuit 15-1 performs the output operation of the FM signal Tx(k) is, for example, k=1, 3, 5, . . .

[0038] When the transmitting circuit 15-2 receives the FM signal Tx(k) from the high frequency signal generating circuit 14 at the timing for outputting the FM signal Tx(k), it amplifies the FM signal Tx(k) and outputs the amplified FM signal Tx(k) to the transmitting antenna element 11-2. The amplified FM signal Tx(k) is output from the transmission circuit 15-2 to the transmission antenna element 11-2, and an FM transmission wave is radiated from the transmission antenna element 11-2 toward the target area. When the transmission circuit 15-2 is outputting the FM signal Tx(k), the transmission circuit 15-1 stops outputting the FM signal Tx(k). The timing at which the transmission circuit 15-2 performs the output operation of the FM signal Tx(k) is, for example, k=2, 4, 6, . . .

[0039] When the transmitting antenna element 11-1 receives the FM signal Tx(k) from the transmitting circuit 15-1, it radiates an FM transmission wave whose frequency transitions over time, as shown in FIG. 7, toward the target area. When the transmitting antenna element 11-2 receives the FM signal Tx(k) from the transmitting circuit 15-2, it radiates an FM transmission wave as shown in FIG. 7 toward the target area. FIG. 7 is an explanatory diagram showing an example of an FM transmission wave and an FM reception wave. The FM transmission waves radiated from the transmitting antenna element 11-1 and the transmitting antenna element 11-2 are reflected by passengers and the like present in the target area.

[0040] As shown in FIG. 7, the FM reception wave, which is a reflected wave of the FM transmission wave reflected by the passengers, etc., is transmitted from the transmitting antenna element 11-1 or the transmitting antenna element 11-2 for t d After the time has elapsed, the signal is received by the receiving antenna elements 12-1 to 12-4. At this time, the frequency of the FM transmission wave and the frequency of the FM reception wave are f d The frequency difference between the FM transmitted chirp signal and the FM received chirp signal is f dincreases in proportion to the distance between the radio wave sensor 21 and a reflecting object such as a passenger. Figure 8 shows the relationship between the sweep time T and the frequency difference f of the FM transmission wave. d FIG.

[0041] When receiving an FM reception wave, which is a reflected wave, the receiving antenna element 12-m (m=1, 2, 3, 4) outputs a reception signal Rx(k) of the FM reception wave to the receiving circuit 16-m. The receiving circuit 16-m generates a frequency difference f between the frequency of the FM signal Tx(k) output from the high frequency signal generating circuit 14 and the frequency of the received signal Rx(k) output from the receiving antenna element 12-m. d Extract (k). The receiving circuit 16-m receives the frequency difference f d (k) m (k) and generate an intermediate frequency signal IF m (k) is output to the A / D conversion circuit 17.

[0042] The A / D conversion circuit 17 receives the intermediate frequency signal IF m (k) receives the intermediate frequency signal IF m (k) is converted from an analog signal to a digital signal D m Convert to (k). The A / D conversion circuit 17 converts the digital signal D m (k) is output to the map generating unit 18.

[0043] The map generating unit 18 receives the digital signal D m Get (k)(m=1,2,3,4). The map generator 18 generates the digital signal D m A motion map is generated based on (k) (m=1,2,3,4).

[0044] An example of the flow up to the generation of the operation map by the map generating unit 18 will be described below. The map generating unit 18 receives the digital signal D m(k) (m=1,2,3,4) digital signal D m (k) is Fourier transformed. The map generator 18 calculates the first frequency spectrum Sp1 by combining the results of the Fourier transform of the four digital signals D1(k) to D4(k). Digital signal D m By Fourier transforming (k), the spectrum value of the received signal Rx(k) (k=1, , K) of the reflected wave from a reflecting object such as a passenger is calculated as the beat frequency Fs 1,n It is accumulated to the signal (n=1,...,N). Beat frequency Fs 1,n The signal strength of 1,n The signal strength of the first frequency spectrum Sp1 is higher than that of the other frequencies, and becomes the peak value of the first frequency spectrum Sp1. TIFF0007756815000001.tif14166In equation (1), R n is the distance between the radio wave sensor 21 and the passenger, and c is the propagation speed of the radio wave.

[0045] The map generating unit 18 selects a beat frequency Fs at which the signal intensity reaches a peak value from the first frequency spectrum Sp1. 1,n Explore. The map generator 18 calculates the beat frequency Fs 1,n By substituting this into equation (1), the distance R n Calculate.

[0046] Each time the map generation unit 18 calculates the first frequency spectrum Sp1, it calculates the second frequency spectrum Sp2 by performing a Fourier transform of the first frequency spectrum Sp1 in the time direction of the FM transmission wave that is periodically radiated from the transmitting antenna 11. The first frequency spectrum Sp1 is Fourier transformed in the time direction of the FM transmission wave, so that the spectral value of the received signal Rx(k) (k=1, . . . , K) of the wave reflected from the reflecting object is calculated based on the relative velocity v between the radio wave sensor 21 and the reflecting object. n The beat frequency Fs shown in the following equation (2) corresponds to2,n is accumulated to Beat frequency Fs 2,n The signal strength of 2,n The signal strength of the second frequency spectrum Sp2 is higher than that of the other frequencies, and becomes the peak value of the second frequency spectrum Sp2. TIFF0007756815000002.tif15166In equation (2), f0 is the center frequency of the FM signal Tx(k).

[0047] The map generating unit 18 selects a beat frequency Fs at which the signal intensity reaches a peak value from the second frequency spectrum Sp2. 2,n Explore. The map generator 18 calculates the beat frequency Fs 2,n By substituting this into equation (2), the relative velocity v n Calculate.

[0048] The installation positions of the receiving antenna elements 12-1 to 12-4 are different from each other in the vehicle width direction. Therefore, even if a wave is reflected from the same reflecting object, the propagation distance to reach receiving antenna element 12-1 is different from the propagation distance to reach receiving antenna element 12-m (m = 2, 3, 4). Therefore, a phase difference Δφ occurs between the phase of the FM received wave received by receiving antenna element 12-1 and the phase of the FM received wave received by receiving antenna element 12-m (m = 2, 3, 4), as shown in the following equation (3). TIFF0007756815000003.tif17166In equation (3), d x is the distance between the receiving antenna elements 12-1 to 12-4 in the direction parallel to the vehicle width direction, and θ x,n is the angle of incidence of the FM reception wave on the xz plane with respect to the receiving antenna element 12-m, and λ is the wavelength of the FM transmission wave.

[0049] The map generating unit 18 receives the digital signal D m (k) (m=1,2,3,4) digital signal D m (k) is Fourier transformed. The map generator 18 generates a third frequency spectrum Sp by combining the Fourier transform results of the four digital signals D1(k) to D4(k). 3,m Calculate. The map generator 18 generates a third frequency spectrum Sp 3,m Each time a third frequency spectrum Sp is calculated, 3,m The fourth frequency spectrum Sp is obtained by Fourier transforming the periodic FM transmission wave in the time direction. 4,m Calculate. The map generator 18 generates a fourth frequency spectrum Sp 4,m Each time a fourth frequency spectrum Sp is calculated, 4,m is Fourier transformed to calculate the fifth frequency spectrum Sp5. The fourth frequency spectrum, Sp 4,m is Fourier transformed across the vehicle width direction (direction parallel to the x-axis), which is the direction in which the receiving antenna elements 12-1 to 12-4 are arranged, the spectral value of the received signal Rx(k) (k=1, . . . , K) of the wave reflected from the reflecting object is expressed as x,n The frequency component Fs corresponding to 5,n is accumulated to Frequency component Fs 5,n The signal strength of the frequency component Fs 5,n The signal intensity of the fifth frequency spectrum Sp5 is higher than that of the other frequency components.

[0050] The map generating unit 18 selects a frequency component Fs where the signal intensity reaches a peak value from the fifth frequency spectrum Sp5. 5,n Explore. The map generator 18 calculates the frequency component Fs 5,n By substituting into the following equation (4), the incident angle θ in the k-th transmission and reception on the xz plane is x,n Calculate. TIFF0007756815000004.tif15166

[0051] The radio wave sensor 21 shown in Fig. 4 has four receiving antenna elements 12-1 to 12-4. The radio wave sensor 21 also has two transmitting antenna elements 11-1 and 11-2, and the transmitting antenna elements 11-1 and 11-2 alternately radiate FM transmission waves. Therefore, as shown in Fig. 9, the radio wave sensor 21 configures virtual antennas 12-1' to 12-8' that are equivalent to the four receiving antenna elements 12-1 to 12-4 being lined up in two rows in the vehicle height direction. In the example of FIG. 9, virtual antennas 12-1' to 12-4' and virtual antennas 12-5' to 12-8' are aligned in the vehicle height direction. The distance between the virtual antennas 12-1' to 12-4' and the virtual antennas 12-5' to 12-8' in the vehicle height direction is d. y is. FIG. 9 is an explanatory diagram showing an example of virtual antennas 12-1' to 12-8' configured in the radio wave sensor 21. As shown in FIG.

[0052] The map generator 18 generates a fourth frequency spectrum Sp 4,m is Fourier transformed in the vehicle height direction (direction parallel to the y-axis), which is the direction in which the virtual antennas 12-1′ etc. and 12-5′ etc. are aligned, to obtain the frequency component Fs 5,n By substituting into the following equation (5), the incident angle θ in the k-th transmission and reception on the yz plane is y,n Calculate. TIFF0007756815000005.tif15166

[0053] Then, the map generating unit 18 calculates the distance R n , the angle of incidence θ in the xz plane x,n and the angle of incidence θ in the yz plane y,nA three-dimensional spatial distribution is generated with the dimensions of each of the above. The process of generating the three-dimensional spatial distribution itself is a known technique, so a detailed description will be omitted. In the first embodiment, the three-dimensional spatial distribution generated by the map generation unit 18 is a "moving object map." The moving object map represents the distribution of areas in the vehicle cabin where moving objects exist in three dimensions. In detail, the moving object map represents the minute movements of moving objects in the vehicle cabin, in other words, objects that have reflected the radio waves irradiated by the radio wave sensor 21, using a plurality of grids that correspond to the reflection points of the radio waves in three-dimensional space.

[0054] 10A, 10B, and 10C are diagrams showing examples of a moving object map generated by the radio wave sensor 21 in the first embodiment. Fig. 10A shows an example of a top view of the moving object map, Fig. 10B shows an example of a side view of the moving object map, and Fig. 10C shows an example of a front view of the moving object map. For ease of explanation, Fig. 10A, Fig. 10B, and Fig. 10C show the moving object map in three views. The moving object maps shown in Fig. 10A, Fig. 10B, and Fig. 10C are moving object maps for the case where an occupant is present in the rear right seat in the vehicle cabin.

[0055] In the moving object map, a numerical value is assigned to each grid according to the speed of an object present at the grid position. Specifically, the faster the speed of an object present at a grid position, the larger the numerical value assigned to that grid. In other words, in the moving object map, a grid included in a range where a moving object with slight movement, in other words, an occupant, is present, is assigned a larger numerical value than a grid where no occupant is present. 10A, 10B, and 10C, the larger the numerical value assigned to a grid, the darker the grid is. In other words, in the moving object maps shown in Figures 10A, 10B, and 10C, the darker the grid is that is included in the area where an occupant is present. In addition, stationary objects, in other words, reflected wave components with a velocity of 0, do not appear in the moving object map.

[0056] Furthermore, each grid in the moving object map is associated with a coordinate in the real space inside the vehicle, which is expressed in a three-dimensional coordinate system that represents the real space inside the vehicle. For example, the map generating unit 18 assumes that the radio wave sensor 21 is installed at a reference sensor position, and associates each grid with coordinates in the real space inside the vehicle cabin based on the reference position.

[0057] The communication circuit 19 transfers the moving object map output from the map generation unit 18 to the information processing device 1 via the interface unit 210, which will be described later. The power supply circuit 200 receives power supply from a control unit (not shown) or the like via an interface unit 210 . The power supply circuit 200 distributes the received power as driving power to the high frequency signal generating circuit 14, the radio wave transmitting unit 15, the radio wave receiving unit 16, the A / D conversion circuit 17, the map generating unit 18 and the communication circuit 19, respectively. The interface unit 210 is an interface for connecting the radio wave sensor circuit unit 13 to the information processing device 1 or a control unit (not shown) or the like.

[0058] As such, the radio wave sensor 21 in embodiment 1 is a radio wave sensor 21 that can detect the distance and angle to an object present in the vehicle cabin and whether the object is a moving object, and is assumed to be a radio wave sensor that can generate a moving object map based on the distance and angle to the detected object (more specifically, the moving object). In the above description, the receiving antenna 12 in the radio wave sensor 21 has four receiving antenna elements 12-1 to 12-4. However, this is merely an example. The receiving antenna 12 may have multiple receiving antenna elements, and the receiving antenna 12 may have two receiving antenna elements, three receiving antenna elements, or five or more receiving antenna elements. Also, in the above description, the transmitting antenna 11 in the radio wave sensor 21 has two transmitting antenna elements 11-1 and 11-2. However, this is merely an example. The transmitting antenna 11 in the radio wave sensor 21 may have only one transmitting antenna element, or may have three or more transmitting antenna elements.

[0059] Returning to FIG. 1, an example of the configuration of the information processing device 1 will be described. The information processing device 1 includes a moving object distribution information acquisition unit 101, an image acquisition unit 102, a camera error detection unit 103, a radio wave sensor error calculation unit 104, a correction unit 105, and an output unit .

[0060] The moving object distribution information acquisition unit 101 acquires moving object distribution information generated by the radio wave sensor 21, that is, a moving object map, from the detection sensor 2. The moving object distribution information acquisition unit 101 outputs the acquired moving object map to the correction unit 105 .

[0061] The image acquisition unit 102 acquires the captured image captured by the camera 22 from the detection sensor 2. The image acquisition unit 102 outputs the acquired captured image to the camera error detection unit 103 .

[0062] The camera error detection unit 103 detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102 . In the first embodiment, the "installation error of camera 22" refers to a deviation in the installation position of camera 22 in the real space inside the vehicle cabin (installation position error) or a deviation in the orientation of camera 22 in the real space inside the vehicle cabin (orientation error of camera 22). More specifically, the "installation error of camera 22" refers to an error in the installation position of camera 22 relative to a predetermined installation position (hereinafter referred to as a "reference camera position"), or an error in the orientation of camera 22 relative to a predetermined orientation (hereinafter referred to as a "reference camera orientation"). The error in the installation position of the camera 22 is, more specifically, an error in the installation position of the camera 22 in the vehicle cabin in the vehicle width direction, vehicle height direction, and vehicle length direction.

[0063] For example, the camera error detection unit 103 uses a known image recognition technique to compare the position and size of a predetermined target object on the captured image with the position (hereinafter referred to as the "target position") and size (hereinafter referred to as the "target size") of the target object on the captured image where the target object should be, thereby detecting an error in the installation position of the camera 22 among the installation errors of the camera 22. The target object may be, for example, a structure inside the vehicle, such as a window frame, a vehicle body, or a seat frame inside the vehicle, or may be a marker that is attached in advance to the vehicle interior by printing or the like. The target position and target size of the target on the captured image are, in detail, the position and size of the target on the captured image captured by camera 22 when camera 22 is installed at the reference camera position, facing the reference camera, and without any misalignment.

[0064] For example, an administrator or the like may conduct a test in advance to obtain information on the target position and target size of the target, and store the obtained information on the target position and target size in the camera error detection unit 103 in association with information indicating the target. The target position and target size are represented by coordinates on the captured image. The target position is represented, for example, by the coordinates of the center of the target in the captured image. The target size is represented, for example, by the coordinates of the four corners of the smallest rectangle that surrounds the target in the captured image.

[0065] The camera error detection unit 103 can detect an error in the vehicle width direction in the real space of the installation position of the camera 22 based on the difference in the vehicle width direction between the position of the target object captured in the captured image acquired by the image acquisition unit 102 and the target position. Note that, for example, a calculation formula for determining the difference in the vehicle width direction between the position of the target object and the target position in the real space when the difference in the vehicle width direction between the position of the target object and the target position in the captured image is large is defined in advance and stored in the camera error detection unit 103.

[0066] Furthermore, the camera error detection unit 103 can detect an error in the vehicle height direction in the real space of the installation position of the camera 22 based on the difference in the vehicle height direction between the position of the target object captured in the captured image acquired by the image acquisition unit 102 and the target position. Note that, for example, a calculation formula for determining the difference in the vehicle height direction between the position of the target object and the target position in the real space when the difference in the vehicle height direction between the position of the target object and the target position in the captured image is large is defined in advance and stored in the camera error detection unit 103.

[0067] Furthermore, the camera error detection unit 103 can detect an error in the vehicle length direction in real space of the installation position of the camera 22 based on the difference between the target size and the size of the target object captured in the captured image acquired by the image acquisition unit 102. Note that a calculation formula for determining, for example, the difference in the vehicle length direction between the target position and the target position in real space when the difference in the vehicle length direction between the target position and the target position in the captured image is large is defined in advance and stored in the camera error detection unit 103.

[0068] In addition, the camera error detection unit 103 can detect an error in the orientation of the camera 22, which is one of the installation errors of the camera 22, based on the difference between the positions of multiple targets captured on the captured image acquired by the image acquisition unit 102 and the target positions of the multiple targets.

[0069] For example, the camera error detection unit 103 may detect an error in the installation position of the camera 22 and an error in the orientation of the camera 22, i.e., an installation error of the camera 22, based on the captured image acquired by the image acquisition unit 102, information indicating the target object, the target position, the target size, and a trained model (hereinafter referred to as a "machine learning model"). The machine learning model is a model that receives the captured image, information indicating the target object, the target position, and the target size as input, and outputs an error in the installation position of the camera 22 and an error in the orientation of the camera 22.

[0070] The camera error detection unit 103 outputs information regarding the detected installation error of the camera 22, i.e., information regarding the error in the installation position of the camera 22 and the error in the orientation of the camera 22 (hereinafter referred to as "camera installation error information") to the radio wave sensor error calculation unit 104. In the camera installation error information, the installation position error of the camera 22 is expressed, for example, by distances in real space within the vehicle cabin (distances in the vehicle width direction, vehicle height direction, and vehicle length direction). The orientation error of the camera 22 is expressed, for example, by an angle indicating the difference in the orientation of the camera 22 from the orientation of a reference camera.

[0071] The radio wave sensor error calculation unit 104 calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103 . As described above, in embodiment 1, the "installation error of radio wave sensor 21" refers to an error in the installation position of radio wave sensor 21 relative to a predetermined installation position (hereinafter referred to as the "reference sensor position") in the real space inside the vehicle cabin, or an error in the orientation of radio wave sensor 21 relative to a predetermined orientation (hereinafter referred to as the "reference sensor orientation"). More specifically, the error in the installation position of the radio wave sensor 21 refers to an error in the installation position of the radio wave sensor 21 in the vehicle width direction, vehicle height direction, and vehicle length direction within the vehicle compartment.

[0072] Since the positional relationship between the radio wave sensor 21 and the camera 22 in the detection sensor 2 and the relationship between the orientation of the radio wave sensor 21 and the orientation of the camera 22 are known, the radio wave sensor error calculation unit 104 can calculate the installation error of the radio wave sensor 21 based on the installation error of the camera 22. The radio wave sensor error calculation unit 104 may calculate the installation error of the radio wave sensor 21, for example, using a table in which the installation error of the camera 22 and the installation error of the radio wave sensor 21 are associated (hereinafter referred to as the "radio wave sensor error calculation table"). The radio wave sensor error calculation table is generated in advance by an administrator or the like and stored in the radio wave sensor error calculation unit 104 .

[0073] The radio wave sensor error calculation unit 104 outputs the calculated information relating to the installation error of the radio wave sensor 21 (hereinafter referred to as “sensor installation error information”) to the correction unit 105. In the sensor installation error information, the installation position error of the radio wave sensor 21 is expressed, for example, by distances in real space within the vehicle cabin (distances in the vehicle width direction, vehicle height direction, and vehicle length direction). The orientation error of the radio wave sensor 21 is expressed, for example, by the difference in orientation of the radio wave sensor 21 from the orientation of a reference sensor.

[0074] The correction unit 105 corrects the moving object map acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. In detail, for example, the correction unit 105 first calculates parameters for correcting the moving object map (hereinafter referred to as “map correction parameters”) based on the installation error of the radio wave sensor 21. The map correction parameters are parameters that define the amount of rotation or horizontal movement of the moving body map depending on, for example, the error in the installation position of the radio wave sensor 21 (i.e., the error in the installation position of the radio wave sensor 21 in the vehicle width direction, the error in the vehicle height direction, and the error in the vehicle length direction) and the error in the orientation of the radio wave sensor 21 (the difference in orientation from the reference sensor orientation).

[0075] The map correction parameters may be, for example, a table (hereinafter referred to as a "map correction table") that converts coordinates associated with each grid in the moving object map. The coordinates associated with each grid are, for example, coordinates in the real space inside the vehicle cabin. More specifically, the map correction table is information that associates the error in the installation position of the radio wave sensor 21 (i.e., the error in the installation position of the radio wave sensor 21 in the vehicle width direction, the error in the vehicle height direction, and the error in the vehicle length direction), the error in the orientation of the radio wave sensor 21 (the difference in orientation from the orientation of the reference sensor), the amount of movement (e.g., how many pixels or how many millimeters) that each grid of the moving object map should be moved in the vehicle width direction, vehicle height direction, and vehicle length direction, and the angle by which the moving object map should be rotated after the movement.

[0076] A plurality of map correction tables may be prepared, for example, depending on the type of installation error of radio wave sensor 21. For example, a plurality of map correction tables may be prepared, such as a map correction table for when there is an error in the installation position of radio wave sensor 21 only in the vehicle width direction, a map correction table for when there is an error only in the vehicle height direction, and a map correction table for when there is an error only in the vehicle length direction. Correction unit 105 determines which map correction table to use to correct the moving body map depending on the type of installation error of radio wave sensor 21.

[0077] The correcting unit 105 outputs the corrected moving object map to the output unit 106. In addition, if correction is not required, in other words, if there is no installation error of the radio wave sensor 21, the correction unit 105 outputs the moving object map acquired by the moving object distribution information acquisition unit 101 as a corrected moving object map to the output unit 106.

[0078] The output unit 106 outputs the corrected moving object map output from the correction unit 105, that is, the moving object map corrected by the correction unit 105, to the monitoring device 3 as error consideration information.

[0079] The monitoring device 3 according to the first embodiment will be described. The monitoring device 3 monitors the occupants in the vehicle cabin based on the moving object map output from the information processing device 1, specifically, the corrected moving object map. The monitoring device 3 includes an occupant detection device 31 and a physique determination device 32 .

[0080] The occupant detection device 31 detects whether a child has been left behind in the vehicle cabin based on the corrected moving object map. In the first embodiment, a child is assumed to be an occupant whose physique makes it difficult for them to get out of the vehicle on their own if left behind in the vehicle. In the first embodiment, children include infants.

[0081] For example, the occupant detection device 31 first detects an occupant present in the vehicle compartment and determines whether the occupant is an adult or a child. Specifically, the occupant detection device 31 determines whether the spatial distribution included in the corrected moving object map (for example, the spatial distribution shown by the colored grid in Figure 10) is a spatial distribution corresponding to an occupant (hereinafter referred to as "occupant spatial distribution") based on the shape of the spatial distribution. Any method can be used for making a judgment based on the shape of the spatial distribution, but one possible method is to use a model that has learned the shape of the occupant spatial distribution to determine whether the spatial distribution included in the moving object map is an occupant spatial distribution. The occupant detection device 31 also calculates the relative speed v corresponding to the spatial distribution included in the corrected moving object map. nis greater than a preset threshold value (hereinafter referred to as a "speed determination threshold value"), the spatial distribution may be determined to be an occupant spatial distribution. n The speed determination threshold value is stored in a location that can be referenced by the occupant detection. Even if the occupant is not moving their arms or legs, they are still breathing and therefore moving to some extent. If the occupant detection device 31 determines that the spatial distribution included in the corrected moving object map is an occupant spatial distribution, it detects that an occupant is present in the vehicle compartment.

[0082] Furthermore, if the occupant detection device 31 determines that the spatial distribution included in the corrected moving object map is an occupant spatial distribution, the occupant detection device 31 calculates the distance R n , incident angle θ x,n and the angle of incidence θ y,n The position of the occupant in the occupant space distribution is identified from each of the distances R n and the incident angle θ with respect to the radio wave sensor 21 x,n and the angle of incidence θ y,n If the distance R from the radio wave sensor 21 is known, the occupant detection device 31 can identify the position of the occupant. n and the incident angle θ with respect to the radio wave sensor 21 x,n and the angle of incidence θ y,n are associated with each other. The seating position of the occupant is indicated, for example, by the seat (for example, rear right seat, rear left seat, or rear center seat).

[0083] Furthermore, the occupant detection device 31 determines whether the detected occupant is an adult or a child for each seat based on the size of the occupant spatial distribution included in the corrected moving object map. Any method may be used for the determination based on the size of the occupant space distribution. For example, a possible mode is to determine whether the occupant is an adult or a child by using a pre-defined virtual area for determining physique (hereinafter referred to as "physique determination area") in the space inside the vehicle cabin. The physique determination area is set in advance by an administrator or the like and stored in a location that can be referenced by the monitoring device 3. The physique determination area is, for example, a rectangular area on the seat surface of each seat, with the bottom surface being a rectangle that includes the seat surface and the height from the seat surface to the top of the headrest. The occupant detection device 31 determines whether the occupant is an adult or a child, for example, based on how high the occupant space distribution is in the physique determination area. Another possible method of determining whether an occupant is an adult or a child is to use a model that has learned the size of the occupant space distribution for adults and children.

[0084] Then, the occupant detection device 31 detects whether a child has been left behind in the vehicle compartment based on the determination result as to whether the detected occupant is an adult or a child. For example, when the occupant detection device 31 determines that there is no adult in the vehicle cabin and that there is a child, that is, when it determines that only a child is present in the vehicle cabin, it detects that a child has been left behind in the vehicle cabin. For example, if an adult is present in the vehicle compartment, or if neither an adult nor a child is present, the occupant detection device 31 detects that no child has been left behind in the vehicle compartment. When the occupant detection device 31 detects that a child has been left behind in the vehicle compartment, it causes an output device (not shown) to output an alarm. The output device is, for example, a mobile terminal carried by the vehicle owner, or a horn or light provided in the vehicle.

[0085] Furthermore, when the occupant detection device 31 detects an occupant present in the vehicle compartment, the occupant detection device 31 may cause the output device to output a sound or the like urging the occupant to fasten their seat belt.

[0086] The physique determination device 32 determines the physique of an occupant present in the vehicle cabin based on the corrected moving object map. For example, the physique of the occupant determined by the physique determination device 32 is either "adult" or "child." In this case, the physical size determination device 32 determines the physical size of the occupant in a manner similar to the method by which the above-mentioned occupant detection device 31 detects an occupant present in the vehicle cabin and determines whether the occupant is an adult or a child.

[0087] Furthermore, for example, the physique determination device 32 may further classify the occupant's physique into categories such as "large adult," "small adult," or "child." In this case, for example, the physique determination region is divided into three regions in the height direction. Let the three regions be, in order from highest to lowest, the first physique determination region, the second physique determination region, and the third physique determination region. The physique determination device 32 determines whether the occupant spatial distribution included in the corrected dynamic object map is included in the first physique determination region, the second physique determination region, or the third physique determination region. For example, if the occupant spatial distribution is included in all of the first physique determination region, the second physique determination region, and the third physique determination region, the physique determination device 32 determines that the occupant is a "large adult." For example, if the occupant spatial distribution is included only in the third physique determination region, the physique determination device 32 determines that the occupant is a "child."

[0088] The physique determination device 32 outputs information relating to the determined physique of the occupant (hereinafter referred to as the "physique determination result") to, for example, a seat belt control device (not shown) and a seat control device (not shown). The seat belt control device controls, for example, the amount of seat belt withdrawal based on the physique determination result output from the physique determination device 32. The seat control device adjusts, for example, the seat position or the backrest based on the physique determination result output from the physique determination device 32.

[0089] The operation of the information processing device 1 according to the first embodiment will be described. FIG. 11 is a flowchart for explaining the operation of the information processing device 1 according to the first embodiment. For example, when the information processing device 1 receives an operation instruction, it performs an operation as shown in the flowchart of Fig. 11. Note that a control unit (not shown) of the information processing device 1 receives the operation instruction, and upon receiving the operation instruction, the control unit operates each component included in the information processing device 1. The operation instruction is input by, for example, an administrator, etc. The administrator, etc. inputs the operation instruction when, for example, installing the detection sensor 2 in a vehicle.

[0090] The moving object distribution information acquisition unit 101 acquires moving object distribution information generated by the radio wave sensor 21, that is, a moving object map, from the detection sensor 2 (step ST1). The moving object distribution information acquisition unit 101 outputs the acquired moving object map to the correction unit 105 .

[0091] The image acquisition unit 102 acquires the captured image captured by the camera 22 from the detection sensor 2 (step ST2). The image acquisition unit 102 outputs the acquired captured image to the camera error detection unit 103 .

[0092] The camera error detection unit 103 detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102 in step ST2 (step ST3). The camera error detection unit 103 outputs the camera installation error information to the radio wave sensor error calculation unit 104 .

[0093] The radio wave sensor error calculation unit 104 calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103 in step ST3 (step ST4). The radio wave sensor error calculation unit 104 outputs the sensor installation error information to the correction unit 105.

[0094] The correction unit 105 corrects the moving object map acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST4 (step ST5). The correcting unit 105 outputs the corrected moving object map to the output unit 106. In addition, if correction is not required, in other words, if there is no installation error of the radio wave sensor 21, the correction unit 105 outputs the moving object map acquired by the moving object distribution information acquisition unit 101 as a corrected moving object map to the output unit 106.

[0095] The output unit 106 outputs the corrected moving object map output from the correcting unit 105 in step ST5, ie, the moving object map corrected by the correcting unit 105, as error consideration information to the monitoring device 3 (step ST6).

[0096] In the flowchart of Fig. 11, the process of step ST1 is performed before the process of step ST2, but this is merely an example. For example, in the flowchart of Fig. 11, the process of step ST1 and the process of step ST2 may be performed in parallel. The process of step ST1 only needs to be performed before the process of step ST5 is performed.

[0097] In this way, the information processing device 1 according to the first embodiment acquires from the detection sensor 2 an image captured by the camera 22, and detects an installation error of the camera 22 based on the acquired image. When the information processing device 1 detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22. The information processing device 1 corrects the moving object distribution information generated by the radio wave sensor 21, i.e., the moving object map, acquired from the detection sensor 2, based on the calculated installation error of the radio wave sensor 21. Then, the information processing device 1 outputs the corrected moving object map to the monitoring device 3 as error consideration information. The monitoring device 3 monitors the interior of the vehicle based on the corrected moving object map output from the information processing device 1.

[0098] As described above, when the radio wave sensor 21 is installed in the vehicle cabin, it is generally fixed to the vehicle cabin via some structure, such as the ceiling or dashboard, between the sensor and the vehicle frame. Therefore, there is a possibility that an installation error occurs in the radio wave sensor 21 installed in the vehicle cabin. If an installation error occurs in the radio wave sensor 21, a deviation in the position and angle of a moving object detected by the radio wave sensor 21 also occurs. In particular, the position of the body or head of an occupant sitting in a seat far from the radio wave sensor 21 will be significantly deviated relative to the radio wave sensor 21, resulting in a large deviation in the position and angle of the occupant detected by the radio wave sensor 21. The deviation in the position and angle of the moving object detected by the radio wave sensor 21 significantly affects the monitoring results of the monitoring device 3, which monitors the occupants using a moving object map generated by the radio wave sensor 21.

[0099] Here, Figures 12A and 12B are figures for explaining an example of the effect that would occur on the monitoring device 3, which monitors occupants using a moving object map generated by the radio wave sensor 21, if there were a discrepancy in the position and angle of the moving object detected by the radio wave sensor 21. 12A and 12B show, as an example, a case in which a deviation in the orientation of the radio wave sensor 21 causes a deviation in the position and angle of the moving object detected by the radio wave sensor 21, which affects the monitoring device 3. Here, the monitoring device 3 is a physique determination device 32, which uses a physique determination area to determine whether the occupant's physique is a "large adult," a "small adult," or a "child." 12A and 12B, the physique determination area is indicated by "S." In addition, in Figures 12A and 12B, the vehicle is indicated by "C," and the range of radio waves emitted by the radio wave sensor 21 is indicated by "R." 12A and 12B are side views of the vehicle. FIG. 12A shows an example of the state inside the vehicle cabin when the radio wave sensor 21 is installed at the same position as the reference sensor so as to radiate radio waves in the direction of the reference sensor. FIG. 12B shows an example of the state inside the vehicle cabin when the position of the radio wave sensor 21 is correct as the reference sensor position, but the orientation of the radio wave sensor 21 is shifted upward from the reference sensor orientation.

[0100] 12B, if there is an upward deviation in the orientation of radio wave sensor 21, the position and angle of the moving object detected by radio wave sensor 21 will also be shifted upward in accordance with the deviation. This is because radio wave sensor 21 calculates the position and angle of the detected moving object assuming that the installation position and orientation of radio wave sensor 21 are the reference sensor position and reference sensor orientation, respectively. If the physique determination device 32 were to determine the physique of the occupant using the dynamic body map generated by the radio wave sensor 21 in a state in which such a deviation has occurred, the physique determination device 32 could erroneously determine the physique of the occupant. This is because the coordinates in the real space inside the vehicle cabin that correspond to each grid on the dynamic body map are shifted by the amount of the deviation. If a physique determination area were to be set in accordance with the coordinates in the real space inside the vehicle cabin that correspond to each grid on the dynamic body map, the physique determination area would have to be set in a range such as that shown in FIG. 12B. However, the physique determination area is actually set based on coordinates in the real space of the vehicle interior without any deviation (for example, FIG. 12A). As a result, the physique determination device 32 may erroneously determine the physique of the occupant, for example, determining the physique of an occupant who should be determined as a "child" as a "small adult."

[0101] Therefore, it is extremely important to detect installation errors of the radio wave sensor 21 and to be able to use the moving object map generated by the radio wave sensor 21 as a moving object map obtained without any installation errors of the radio wave sensor 21. To address this problem, as described above, the information processing device 1 according to embodiment 1 acquires, from the detection sensor 2, an image captured by the camera 22, and detects an installation error of the camera 22 based on the acquired image. Upon detecting an installation error of the camera 22, the information processing device 1 calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22. The information processing device 1 corrects the moving object map acquired from the detection sensor 2 and generated by the radio wave sensor 21, based on the calculated installation error of the radio wave sensor 21. The information processing device 1 then outputs the corrected moving object map to the monitoring device 3 as error consideration information. As a result, the information processing device 1 can provide error-taking information, specifically, a corrected moving object map, to use the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21. The monitoring device 3 can monitor the occupants using the corrected moving object map provided by the information processing device 1, and as a result, the monitoring device 3 can prevent incorrect monitoring of the occupants (for example, false detection of a child being left behind or false determination of the occupant's physique).

[0102] In addition, in embodiment 1, the radio wave sensor 21 is mounted on the detection sensor 2, and the detection sensor 2 includes a fixed part 23, a camera 22 provided on the fixed part 23, and the radio wave sensor 21 provided on the fixed part 23, and when installed in the vehicle cabin, the detection sensor 2 has at least two of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21, or the detection sensor 2 includes the camera 22, the radio wave sensor 21, and a substrate 24 on which the transmitting antenna 11 and receiving antenna 12 of the camera 22 and the radio wave sensor 21 are provided on a common surface. The information processing device 1 acquires an image captured by the camera 22 from such a detection sensor 2, and detects an installation error of the camera 22 based on the acquired image. When the information processing device 1 detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22. The information processing device 1 corrects the moving object map acquired from the detection sensor 2 and generated by the radio wave sensor 21 based on the calculated installation error of the radio wave sensor 21. Then, the information processing device 1 outputs the corrected moving object map to the monitoring device 3 as error consideration information. As a result, the information processing device 1 can provide error-taking information, specifically, a corrected moving object map, to use the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21. The monitoring device 3 can monitor the occupants using the corrected moving object map provided by the information processing device 1, and as a result, the monitoring device 3 can prevent incorrect monitoring of the occupants (for example, false detection of a child being left behind or false determination of the occupant's physique).

[0103] As mentioned above, a technique is generally known in which a marker whose exact direction or distance from the camera 22 is known is captured by the camera 22, and deviations from the position and direction in the captured image that should have been captured are detected, and the installation position and direction of the camera 22 are adjusted or the captured image is corrected. For example, with regard to the radio wave sensor 21, it is not impossible to place an object serving as a marker whose exact direction or distance from the radio wave sensor 21 is known, detect the object, detect a deviation from the object's intended position and direction, and adjust the installation position of the radio wave sensor 21 and the direction of radio wave emission. However, it is not easy to adjust the radio wave sensor 21 in this way inside a vehicle. This is because, for example, it is difficult for the radio wave sensor 21 to distinguish between the marker and other structures in the vehicle interior. Also, with the radio wave sensor 21, a so-called multipath problem occurs, in which the marker appears double when the emitted radio waves are reflected by structures in the vehicle interior. The information processing device 1 of embodiment 1 is not affected by the difficulty of the radio wave sensor 21 in distinguishing between markers and structures other than the markers inside the vehicle cabin, or by the so-called multipath problem that occurs in the radio wave sensor 21, and is therefore able to use the moving object map as a moving object map obtained without any installation error of the radio wave sensor 21.

[0104] In the first embodiment, the information processing device 1 is provided outside the detection sensor 2, but this is merely an example. The information processing device 1 may be provided in the detection sensor 2.

[0105] In addition, in the above-described embodiment 1, the information processing device 1 is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) are provided in the in-vehicle device. Without being limited to this, the system may be configured with an on-board device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, correction unit 105, output unit 106, and a control unit (not shown) being mounted on the on-board device of the vehicle, and the rest being provided on a server connected to the on-board device via a network. In addition, the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, correction unit 105, output unit 106, and a control unit (not shown) may all be provided on the server.

[0106] 2, the detection sensor 2 is configured so that, when the detection sensor 2 is installed in the vehicle cabin, at least two of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21. However, this is merely an example, and it is not essential that, in the detection sensor 2, at least two of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21. However, since the detection sensor 2 is configured so that at least two of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21, the information processing device 1 can easily correct the moving object map compared to a configuration in which only one or none of the x-axis, y-axis, and z-axis specific to the camera 22 coincide with the x-axis, y-axis, and z-axis specific to the radio wave sensor 21.

[0107] In the first embodiment described above, when the detection sensor 2 has the configuration shown in Fig. 3, the camera 22 is configured to be provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24. However, this is merely an example, and it is not essential that the camera 22 be provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24 in the detection sensor 2. However, by configuring the detection sensor 2 so that the camera 22 is provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24, the information processing device 1 can more easily correct the moving object map compared to a configuration in which the camera 22 is not provided between the transmitting antenna 11 and the receiving antenna 12 on the common surface of the substrate 24.

[0108] 13A and 13B are diagrams illustrating an example of a hardware configuration of the information processing device 1 according to the first embodiment. In the first embodiment, the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) are realized by the processing circuit 1001. That is, the information processing device 1 includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained without an installation error of the radio wave sensor 21. The processing circuit 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 13B.

[0109] When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0110] When the processing circuit is the processor 1004, the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005, thereby executing the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown). That is, the information processing device 1 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST1 to ST6 of FIG. 11 described above. The program stored in the memory 1005 can also be said to cause the computer to execute the procedures or methods of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown). Here, the memory 1005 corresponds to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0111] It is also possible to realize some of the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) by using dedicated hardware and some by using software or firmware. For example, the functions of the moving object distribution information acquisition unit 101 and the image acquisition unit 102 can be realized by a processing circuit 1001 as dedicated hardware, and the functions of the camera error detection unit 103, the radio wave sensor error calculation unit 104, the correction unit 105, the output unit 106, and a control unit (not shown) can be realized by having the processor 1004 read and execute programs stored in the memory 1005. The information processing device 1 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the detection sensor 2 or the monitoring device 3.

[0112] As described above, according to the first embodiment, the information processing device 1 is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2 having the camera 22 that is installed in the vehicle cabin and captures an image of a target area in the vehicle cabin where at least an occupant may be present, and the radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of areas in the vehicle cabin where moving objects are present, based on the radio waves that are emitted toward the target area in the vehicle cabin and reflected by objects in the vehicle cabin; a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102; a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103; and an output unit 106 that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. Therefore, the information processing device 1 can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0113] In detail, the information processing device 1 includes a moving object distribution information acquisition unit 101 that acquires moving object distribution information from the detection sensor 2, and a correction unit 105 that corrects the moving object distribution information acquired by the moving object distribution information acquisition unit 101 based on the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and the output unit 106 is configured to output the moving object distribution information corrected by the correction unit 105 (corrected moving object map) as error-considered information. Therefore, the information processing device 1 can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0114] Furthermore, as described above, according to the first embodiment, the detection sensor 2 is provided in a vehicle interior and includes the fixed part 23, the camera 22 provided on the fixed part 23 and capturing an image of a target area in the vehicle interior where at least an occupant may be present, the radio wave sensor 21 provided on the fixed part 23 and configured to generate moving object distribution information that three-dimensionally represents a distribution of an area where moving objects exist in the vehicle interior based on radio waves that are emitted toward the target area in the vehicle interior and reflected by objects in the vehicle interior, the image acquisition unit 102 that acquires the captured image captured by the camera 22, and the camera error detection unit 103 that detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102. The radio wave sensor error calculation unit 104 calculates the installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106 outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104.When installed in the vehicle cabin, the radio wave sensor 21 can be configured so that at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) coincide. Therefore, the detection sensor 2 can provide error-considered information for using the information generated by the radio wave sensor 21, that is, the moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.

[0115] Furthermore, as described above, according to the first embodiment, the detection sensor 2 is provided in the vehicle interior and includes a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle interior based on radio waves that are radiated toward the target area in the vehicle interior and reflected by objects in the vehicle interior, the radio wave sensor 21 having a transmitting antenna 11 that radiates radio waves and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and the camera 22, the transmitting antenna 11, and the receiving antenna 12. 2 are provided on a common surface, an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image captured by the image acquisition unit 102, a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103, and an output unit 106 that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. Therefore, the detection sensor 2 can provide error-considered information for using the information generated by the radio wave sensor 21, that is, the moving object distribution information, as information obtained without any installation error of the radio wave sensor 21.

[0116] Furthermore, according to the first embodiment, the information processing system 4 includes a detection sensor 2 that is provided in a vehicle interior and has a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle interior based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image captured by the image acquisition unit 102, and a radio wave sensor error detection unit 103 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103. The information processing device 1 has a calculation unit 104 and an output unit 106 that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is installed in a vehicle cabin and has a fixed part 23, a camera 22 installed on the fixed part 23, and a radio wave sensor 21 installed on the fixed part 23, and is characterized in that when installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) are coincident. Therefore, the information processing system 4 can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0117] Furthermore, according to the first embodiment, the information processing system 4 is provided in a vehicle interior, and includes a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle interior based on a radio wave that is emitted toward the target area in the vehicle interior and reflected by an object in the vehicle interior. The information processing system 4 includes an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a detection unit 104 that detects an installation error of the camera 22 detected by the camera error detection unit 103. and an output unit 106 that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104; and a detection sensor 2 that is provided in the vehicle cabin and has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are provided on a common surface. Therefore, the information processing system 4 can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0118] Embodiment 2 In the first embodiment, the information processing device has a function of correcting a moving object map, and outputs a corrected moving object map as error consideration information. In the second embodiment, an embodiment will be described in which the information processing device does not have a function of correcting the moving object map.

[0119] FIG. 14 is a diagram illustrating an example of the configuration of an information processing device 1a according to the second embodiment. In the second embodiment, an information processing device 1a is connected to a detection sensor 2 and a monitoring device 3. The information processing device 1a and the detection sensor 2 constitute an information processing system 4a. In the second embodiment, the information processing device 1a, the detection sensor 2, and the monitoring device 3 are mounted on, for example, a vehicle (not shown).

[0120] The detection sensor 2 according to the second embodiment has the same configuration as the detection sensor 2 according to the first embodiment described with reference to FIGS. Furthermore, the monitoring device 3 includes an occupant detection device 31 and a physique determination device 32, similar to the monitoring device 3 described in the first embodiment. 14, the same components as those of the information processing device 1 described in the first embodiment with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. The information processing device 1a according to the second embodiment differs from the information processing device 1 according to the first embodiment in that it does not include a correction unit 105. Furthermore, in the information processing device 1a according to the second embodiment, the specific operation of the output unit 106a is different from the specific operation of the output unit 106 in the information processing device 1 according to the first embodiment.

[0121] In embodiment 2, the output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 and the sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 to the monitoring device 3 as error consideration information. That is, the output unit 106a associates the moving object distribution information output from the moving object distribution information acquisition unit 101 with the sensor installation error information output from the radio wave sensor error calculation unit 104, and outputs the result to the monitoring device 3 as error consideration information. In the second embodiment, the moving object distribution information acquisition unit 101 outputs the acquired moving object distribution information to the output unit 106a. In the second embodiment, the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106a.

[0122] In the second embodiment, the information processing device 1a does not have a function to correct the moving object map. Therefore, the moving object map included in the error consideration information output by the information processing device 1a to the monitoring device 3 may be a moving object map generated by the radio wave sensor 21 in a state where there is an installation error of the radio wave sensor 21. Therefore, in embodiment 2, when monitoring an occupant, the monitoring device 3 performs processing (hereinafter referred to as "monitoring correction processing") based on the sensor installation error information contained in the error consideration information so that the moving object map can be used as a moving object map generated without any installation error of the radio wave sensor 21, and then monitors the occupant. An example of the monitoring correction process is a process of correcting a region for determining physique. For example, the occupant detection device 31 and the physique determination device 32 correct the position or orientation of the physique determination region in accordance with the installation error of the radio wave sensor 21 based on the sensor installation error information. This allows the monitoring device 3 to monitor occupants using the moving object map generated by the radio wave sensor 21 provided by the information processing device 1a as a moving object map generated without any installation error of the radio wave sensor 21, and as a result, the monitoring device 3 can prevent incorrect monitoring of occupants (for example, false detection of a child being left behind or false determination of the physique of an occupant).

[0123] The operation of the information processing device 1a according to the second embodiment will be described. FIG. 15 is a flowchart for explaining the operation of the information processing device 1a according to the second embodiment. For example, when the information processing device 1a receives an operation instruction, it performs an operation as shown in the flowchart of Fig. 15. Note that a control unit (not shown) of the information processing device 1a receives the operation instruction, and upon receiving the operation instruction, the control unit operates each component included in the information processing device 1a. The operation instruction is input by, for example, an administrator, etc. The administrator, etc. inputs the operation instruction when, for example, installing the detection sensor 2 in a vehicle.

[0124] In the flowchart of Figure 15, the specific contents of the processing of steps ST11 to ST14 are the same as the specific contents of the processing of steps ST1 to ST4 in the flowchart of Figure 11, which have already been explained in embodiment 1, so duplicate explanations will be omitted. However, in step ST11, the moving object distribution information acquisition unit 101 outputs the acquired moving object map to the output unit 106a. Also, in step ST14, the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106a.

[0125] The output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 in step ST11 and the sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST14 as error consideration information to the monitoring device 3 (step ST15).

[0126] In the flowchart of Fig. 15, the process of step ST11 is performed before the process of step ST12, but this is merely an example. For example, in the flowchart of Fig. 15, the process of step ST11 and the process of step ST12 may be performed in parallel. It is sufficient that the process of step ST11 is performed before the process of step ST15 is performed.

[0127] In this way, the information processing device 1a according to the second embodiment acquires from the detection sensor 2 an image captured by the camera 22, and detects an installation error of the camera 22 based on the acquired image. When the information processing device 1a detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22. The information processing device 1a outputs, as error consideration information, to the monitoring device 3 the moving object distribution information generated by the radio wave sensor 21 acquired from the detection sensor 2, i.e., the moving object map, and sensor installation error information indicating the calculated installation error of the radio wave sensor 21. As a result, the information processing device 1a can provide error consideration information, specifically, in this case, a moving object map and sensor installation error information, to use the moving object map information generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21. Based on the moving object map and sensor installation error information provided by the information processing device 1a, the monitoring device 3 performs a monitoring correction process to enable the moving object map to be used as a moving object map generated without installation error of the radio wave sensor 21, and then monitors the occupants. As a result, the monitoring device 3 can prevent erroneous monitoring of occupants (for example, erroneous detection of a child being left behind or erroneous determination of the physique of an occupant).

[0128] In the second embodiment, the information processing device 1a is provided outside the detection sensor 2, but this is merely an example. The information processing device 1a may be provided in the detection sensor 2.

[0129] In addition, in the above-described second embodiment, the information processing device 1a is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown) are provided in the in-vehicle device. Without being limited to this, the system may be configured with an on-board device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, output unit 106a, and a control unit (not shown) being mounted on the on-board device of the vehicle, and the rest being provided in a server connected to the on-board device via a network. Furthermore, the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown) may all be provided in the server.

[0130] The hardware configuration of the information processing device 1a according to the second embodiment is the same as the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 13A and 13B, and therefore is not shown in the drawings. In the second embodiment, the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown) are realized by a processing circuit 1001. That is, the information processing device 1a includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained without an installation error of the radio wave sensor 21. The processing circuit 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 that executes a program stored in a memory 1005 as shown in FIG. 13B.

[0131] The processing circuit 1001 reads and executes a program stored in the memory 1005, thereby executing the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown). That is, the information processing device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST11 to ST15 in FIG. 15 described above. The program stored in the memory 1005 can also be said to cause a computer to execute the processing procedures or methods of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106a, and a control unit (not shown). The information processing device 1a includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as a detection sensor 2 or a monitoring device 3.

[0132] As described above, according to the second embodiment, the information processing device 1a is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from a detection sensor 2 having the camera 22 that is provided in the vehicle cabin and captures an image of a target area in the vehicle cabin where at least an occupant may be present, and the radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area in the vehicle cabin where moving objects are present, based on the radio waves that are emitted toward the target area in the vehicle cabin and reflected by objects in the vehicle cabin; a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102; a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103; and an output unit 106a that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. Therefore, the information processing device 1a can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0133] In detail, the information processing device 1a includes a moving object distribution information acquisition unit 101 that acquires moving object distribution information from the detection sensor 2, and the output unit 106a outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 and sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 as error consideration information. Therefore, the information processing device 1a can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0134] Furthermore, according to the second embodiment, the information processing system 4a is provided with a camera 22 that is provided in a vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle interior based on a radio wave radiated toward the target area in the vehicle interior and reflected by an object in the vehicle interior, and the information processing system 4a is provided with an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image captured by the image acquisition unit 102, and a radio wave sensor error detection unit 103 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103. The information processing device 1a has a calculation unit 104 and an output unit 106a that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is installed in a vehicle cabin and has a fixed part 23, a camera 22 installed on the fixed part 23, and a radio wave sensor 21 installed on the fixed part 23, and is characterized in that when installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera 22 (x-axis, y-axis, and z-axis specific to the camera 22) and the three-dimensional coordinate axes of the radio wave sensor 21 (x-axis, y-axis, and z-axis specific to the radio wave sensor 21) are coincident. Therefore, the information processing system 4a can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0135] Furthermore, according to the second embodiment, the information processing system 4a is provided in a vehicle interior, and includes a camera 22 that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle interior based on a radio wave that is emitted toward the target area in the vehicle interior and reflected by an object in the vehicle interior. The information processing system 4a includes an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2, a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102, and a detection unit 104 that detects an installation error of the camera 22 based on the installation error of the camera 22 detected by the camera error detection unit 103. Based on this, the information processing device 1a is configured to include a radio wave sensor error calculation unit 104 that calculates the installation error of the radio wave sensor 21 and an output unit 106a that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2 that is installed inside the vehicle cabin and has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are installed on a common surface. Therefore, the information processing system 4a can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0136] Embodiment 3 In the first embodiment, the information processing device has a function of correcting a moving object map, and outputs a corrected moving object map as error consideration information. In the third embodiment, another embodiment will be described in which the information processing device does not have the function of correcting the moving object map.

[0137] FIG. 16 is a diagram illustrating an example of the configuration of an information processing device 1b according to the third embodiment. In the third embodiment, an information processing device 1b is connected to a detection sensor 2a and a monitoring device 3. The information processing device 1b and the detection sensor 2a constitute an information processing system 4b. In the third embodiment, the information processing device 1b, the detection sensor 2a, and the monitoring device 3 are mounted on, for example, a vehicle (not shown).

[0138] 2 and 3 in the first embodiment, the detection sensor 2a according to the second embodiment includes a control unit 25 that changes the orientation of the radio wave sensor 21, i.e., the direction of radio wave emission by the radio wave sensor 21, based on instruction information output from the information processing device 1b. Details of the instruction information will be described later. In the third embodiment, the detection sensor 2a includes, for example, a servo motor (not shown) that changes the orientation of the radio wave sensor 21. The control unit 25 changes the orientation of the radio wave sensor 21 by controlling the servo motor based on instruction information output from the information processing device 1b. Furthermore, for example, in the detection sensor 2a, the radio wave sensor 21 may be a radio wave sensor 21 that can change the direction of radio wave emission by applying signals of different phases to multiple antennas, such as a phased array antenna. In this case, the control unit 25 instructs the radio wave sensor 21 to change the direction of radio wave emission based on instruction information output from the information processing device 1b. The control unit 25 corrects the installation error of the radio wave sensor 21 by changing the orientation of the radio wave sensor 21 based on the instruction information output from the information processing device 1b. As a result, the moving object map generated by the radio wave sensor 21 without any installation error of the radio wave sensor 21 is output from the detection sensor 2a to the information processing device 1b.

[0139] In the third embodiment, the monitoring device 3 includes an occupant detection device 31 and a physique determination device 32, similar to the monitoring device 3 described in the first embodiment.

[0140] In FIG. 16, the same components as those of the information processing device 1 described in the first embodiment are denoted by the same reference numerals, and redundant description will be omitted. The information processing device 1b according to the third embodiment differs from the information processing device 1 according to the first embodiment in that it does not include a correction unit 105. Furthermore, in the information processing device 1b according to the third embodiment, the specific operation of the output unit 106b is different from the specific operation of the output unit 106 in the information processing device 1 according to the first embodiment.

[0141] In the third embodiment, the output unit 106b outputs instruction information to instruct the detection sensor 2a to change the orientation of the radio wave sensor 21 as error consideration information based on the sensor installation error information output from the radio wave sensor error calculation unit 104. The instruction information includes information that allows the installation error of the radio wave sensor 21 to be determined. Moreover, the output unit 106b outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 to the monitoring device 3. In the third embodiment, the moving object distribution information acquisition unit 101 outputs the acquired moving object distribution information to the output unit 106b. In the third embodiment, the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106b.

[0142] In the third embodiment, the information processing device 1b does not have a function for correcting the moving object map. Instead, the information processing device 1b notifies the detection sensor 2a of how much the installation position or the orientation of the radio wave sensor 21 needs to be changed so that the radio wave sensor 21 is in a state free from installation errors, and outputs instruction information for bringing the radio wave sensor 21 into a state free from installation errors. This allows the information processing device 1b to acquire the moving object map generated by the radio wave sensor 21 as a moving object map obtained without any installation error of the radio wave sensor 21. As a result, the moving object map output by the information processing device 1b to the monitoring device 3 is also a moving object map obtained without any installation error of the radio wave sensor 21. Therefore, the monitoring device 3 can prevent erroneous monitoring of the occupant (for example, erroneous detection of a child being left behind or erroneous determination of the physique of the occupant).

[0143] The operation of the information processing device 1b according to the third embodiment will be described. FIG. 17 is a flowchart for explaining the operation of the information processing device 1b according to the third embodiment. For example, when the information processing device 1b receives an operation instruction, it performs an operation as shown in the flowchart of Fig. 17. Note that a control unit (not shown) of the information processing device 1b receives the operation instruction, and upon receiving the operation instruction, the control unit operates each component included in the information processing device 1b. The operation instruction is input by, for example, an administrator, etc. The administrator, etc. inputs the operation instruction when, for example, installing the detection sensor 2a in a vehicle.

[0144] In the flowchart of Figure 17, the specific contents of the processing of steps ST111 to ST114 are the same as the specific contents of the processing of steps ST1 to ST4 in the flowchart of Figure 11, which have already been explained in embodiment 1, so duplicate explanations will be omitted. However, in step ST111, the moving object distribution information acquisition unit 101 outputs the acquired moving object map to the output unit 106b. Also, in step ST114, the radio wave sensor error calculation unit 104 outputs the sensor installation error information to the output unit 106b.

[0145] The output unit 106b outputs instruction information to the detection sensor 2a to change the orientation of the radio wave sensor 21 as error consideration information based on sensor installation error information indicating the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104 in step ST114 (step ST115-1). Furthermore, the output unit 106b outputs the moving object distribution information acquired by the moving object distribution information acquisition unit 101 in step ST111, that is, the moving object map, to the monitoring device 3 (step ST115-2).

[0146] 17, the process of step ST111 is performed before the process of step ST112, but this is merely an example. For example, in the flowchart of FIG. 17, the process of step ST111 and the process of step ST112 may be performed in parallel. The process of step ST111 only needs to be performed before the process of step ST115-2 is performed. 17, the process of step ST115-1 and the process of step ST115-2 are performed in parallel, but this is merely an example. For example, in the flowchart of FIG. 17, the process of step ST115-1 and the process of step ST115-2 may be performed in sequence.

[0147] Furthermore, with regard to the operation of the information processing device 1b described using the flowchart of Figure 17, for example, in the information processing device 1b, the output unit 106b may output instruction information to the detection sensor 2a, and then output the moving object map generated after the installation error of the radio wave sensor 21 is corrected by the detection sensor 2a, and then output the moving object map to the monitoring device 3. In this case, for example, when the installation error of the radio wave sensor 21 in the detection sensor 2a is corrected, the control unit 25 outputs completion information indicating that the correction has been completed to the output unit 106b of the information processing device 1b. After outputting the instruction information to the detection sensor 2a, the output unit 106b suspends output of the moving object map to the monitoring device 3 until it acquires the completion information from the detection sensor 2a, and starts outputting the moving object map to the monitoring device 3 upon acquiring the completion information from the detection sensor 2a. This enables the information processing device 1b to further reduce the occurrence of erroneous occupant monitoring by the monitoring device 3 (for example, erroneous detection of a child being left behind or erroneous determination of the physique of an occupant).

[0148] In this way, the information processing device 1b according to the third embodiment acquires from the detection sensor 2a an image captured by the camera 22, and detects an installation error of the camera 22 based on the captured image. When the information processing device 1a detects an installation error of the camera 22, it calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22. Based on the calculated installation error of the radio wave sensor 21, the information processing device 1b outputs, as error consideration information, instruction information to the detection sensor 2a to change the direction of radio wave emission by the radio wave sensor 21. As a result, the information processing device 1b can provide error consideration information for using the information of the moving object map generated by the radio wave sensor 21 as information obtained without any installation error of the radio wave sensor 21, specifically, in this case, instruction information for instructing a change in the orientation of the radio wave sensor 21. The monitoring device 3 can monitor the occupants using the moving object map provided by the information processing device 1b and generated without any installation error of the radio wave sensor 21. As a result, the monitoring device 3 can prevent erroneous monitoring of the occupants (for example, erroneous detection of a child being left behind or erroneous determination of the physique of the occupant).

[0149] In the third embodiment, the information processing device 1b is provided outside the detection sensor 2a, but this is merely an example. The information processing device 1b may be provided in the detection sensor 2a.

[0150] In addition, in the above-described third embodiment, the information processing device 1b is an in-vehicle device mounted on a vehicle, and the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown) are provided in the in-vehicle device. Without being limited to this, the system may be configured with an on-board device and a server, with some of the moving object distribution information acquisition unit 101, image acquisition unit 102, camera error detection unit 103, radio wave sensor error calculation unit 104, output unit 106b, and a control unit (not shown) being mounted on the on-board device of the vehicle, and the rest being provided in a server connected to the on-board device via a network. Furthermore, the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown) may all be provided in the server.

[0151] In the third embodiment, the monitoring device 3 acquires the moving object map generated by the radio wave sensor 21 via the information processing device 1b, but this is merely an example. For example, the monitoring device 3 may acquire the moving object map directly from the detection sensor 2a. In this case, in the information processing device 1b, the output unit 106b does not output the moving object map to the monitoring device 3. Furthermore, the information processing device 1b may be configured without the moving object distribution information acquisition unit 101. If the information processing device 1b is configured without the moving object distribution information acquisition unit 101, the processing of step ST111 and the processing of step ST115-2 can be omitted in the operation of the information processing device 1b described using the flowchart of FIG.

[0152] The hardware configuration of the information processing device 1b according to the third embodiment is the same as the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 13A and 13B, and therefore is not shown in the drawings. In the third embodiment, the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown) are realized by a processing circuit 1001. That is, the information processing device 1b includes the processing circuit 1001 for performing control to provide error consideration information for using the moving object map information generated by the radio wave sensor 21 as information obtained without an installation error of the radio wave sensor 21. The processing circuit 1001 may be dedicated hardware as shown in FIG. 13A, or may be a processor 1004 that executes a program stored in a memory 1005 as shown in FIG. 13B.

[0153] The processing circuit 1001 reads and executes a program stored in the memory 1005, thereby executing the functions of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown). That is, the information processing device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST111 to ST115-1 and ST115-2 in FIG. 17 described above. The program stored in the memory 1005 can also be said to cause a computer to execute the processing procedures or methods of the moving object distribution information acquisition unit 101, the image acquisition unit 102, the camera error detection unit 103, the radio wave sensor error calculation unit 104, the output unit 106b, and a control unit (not shown). The information processing device 1b includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the detection sensor 2a or the monitoring device 3.

[0154] As described above, according to the third embodiment, the information processing device 1b is configured to include an image acquisition unit 102 that acquires an image captured by the camera 22 from the detection sensor 2a, which has the camera 22 that is installed in the vehicle cabin and captures an image of a target area in the vehicle cabin where at least an occupant may be present, and the radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of areas in the vehicle cabin where moving objects are present, based on the radio waves that are emitted toward the target area in the vehicle cabin and reflected by objects in the vehicle cabin; a camera error detection unit 103 that detects an installation error of the camera 22 based on the image acquired by the image acquisition unit 102; a radio wave sensor error calculation unit 104 that calculates an installation error of the radio wave sensor 21 based on the installation error of the camera 22 detected by the camera error detection unit 103; and an output unit 106b that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104. Therefore, the information processing device 1b can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0155] Specifically, in the information processing device 1b, the output unit 106b outputs, as error consideration information, instruction information that instructs the detection sensor 2a to change the direction of radio wave radiation from the radio wave sensor 21. Therefore, the information processing device 1b can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0156] Furthermore, according to the third embodiment, the information processing system 4b is provided with a detection sensor 2a including a camera 22 that captures an image of a target area in the vehicle cabin where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle cabin based on a radio wave radiated toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin, and the detection sensor 2a includes an image acquisition unit 102 that acquires the captured image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle cabin based on a radio wave radiated toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin. The information processing device 1b has a radio wave sensor error calculation unit 104 that calculates an installation error, and an output unit 106b that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and the detection sensor 2a is installed in the vehicle cabin and has a fixed part 23, a camera 22 installed on the fixed part 23, and a radio wave sensor 21 installed on the fixed part 23, and is characterized in that when installed in the vehicle cabin, at least two of the positions of the camera 22 and the radio wave sensor 21 in the width direction, height direction, or depth direction in the three-dimensional space within the vehicle cabin coincide. Therefore, the information processing system 4b can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0157] Furthermore, according to the third embodiment, the information processing system 4b is provided with a detection sensor 2a including a camera 22 that captures an image of a target area in the vehicle cabin where at least an occupant may be present, and a radio wave sensor 21 that generates moving object distribution information (moving object map) that three-dimensionally represents the distribution of an area where moving objects exist in the vehicle cabin based on a radio wave that is emitted toward the target area in the vehicle cabin and reflected by an object in the vehicle cabin. The information processing system 4b includes an image acquisition unit 102 that acquires an image captured by the camera 22, a camera error detection unit 103 that detects an installation error of the camera 22 based on the captured image acquired by the image acquisition unit 102, and a detection unit 21 that detects an installation error of the camera 22 based on the installation error of the camera 22 detected by the camera error detection unit 103. Based on this, the information processing device 1b is configured to include a radio wave sensor error calculation unit 104 that calculates the installation error of the radio wave sensor 21 and an output unit 106b that outputs error consideration information to convert the moving object distribution information generated by the radio wave sensor 21 into moving object distribution information that takes into account the installation error of the radio wave sensor 21 calculated by the radio wave sensor error calculation unit 104, and a detection sensor 2a that is installed inside the vehicle cabin and has a radio wave sensor 21 that has a camera 22, a transmitting antenna 11 that radiates radio waves, and a receiving antenna 12 that receives reflected waves of the radio waves radiated from the transmitting antenna 11, and a substrate 24 on which the camera 22, the transmitting antenna 11, and the receiving antenna 12 are installed on a common surface. Therefore, the information processing system 4b can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0158] In detail, in the information processing system 4b, the output unit 106b of the information processing device 1b outputs instruction information to the detection sensor 2a to instruct the radio wave sensor 21 to change the direction of radio wave radiation, and the detection sensor 2a is configured to include a control unit 25 that changes the direction of radio wave radiation by the radio wave sensor 21 based on the instruction information output from the output unit 106b. Therefore, the information processing system 4b can provide error-taking information that allows the information generated based on the reflected waves received by the radio wave sensor 21, which is installed in the vehicle cabin and can detect the position and movement of moving objects within the vehicle cabin by emitting radio waves into the vehicle cabin and receiving reflected waves of the radiated radio waves reflected by objects, i.e., moving object distribution information, to be used as information obtained without any installation error of the radio wave sensor 21.

[0159] In the above first to third embodiments, the occupant detection device 31 or the physique determination device 32 may be configured to change the threshold value used when determining the physique of a moving object, in other words, an occupant, depending on the installation error of the radio wave sensor 21. Specifically, the information processing devices 1, 1a, 1b include information capable of identifying an installation error of the radio wave sensor 21 in the error consideration information for the occupant detection device 31 and the physique determination device 32. The occupant detection device 31 or the physique determination device 32 identifies an installation error of the radio wave sensor 21 from the information capable of identifying an installation error of the radio wave sensor 21 output from the information processing device 1, 1a, 1b, and makes changes such as increasing or decreasing the threshold value used when determining the occupant's physique in accordance with the installation error. This allows the occupant detection device 31 or the physique determination device 32 to determine the physique of the occupant taking into consideration the fact that the moving body map containing errors was originally output from the radio wave sensor 21. More specifically, if there is an installation error in the radio wave sensor 21, the characteristics of the moving body reflection will change, and therefore the occupant detection device 31 or the physique determination device 32 can determine the physique of the occupant taking into consideration the fact that errors will also occur in the information associated with the grids on the moving body map.

[0160] In the above first to third embodiments, the monitoring device 3 includes the occupant detection device 31 and the physique determination device 32, but this is merely an example. The monitoring device 3 may include either the occupant detection device 31 or the physique determination device 32. Furthermore, in the above first to third embodiments, the information processing devices 1, 1a, and 1b can also provide error consideration information to devices other than the monitoring device 3.

[0161] Furthermore, the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. [Industrial Applicability]

[0162] The information processing device of the present disclosure can provide error-taking information that allows information generated based on the reflected waves received by a radio wave sensor that is installed in the vehicle cabin, emits radio waves into the vehicle cabin, and receives reflected waves from objects to detect the position and movement of moving objects in the vehicle cabin, as information obtained without any installation error of the radio wave sensor. [Explanation of symbols]

[0163] 1, 1a, 1b Information processing device, 101 Moving object distribution information acquisition unit, 102 Image acquisition unit, 103 Camera error detection unit, 104 Radio wave sensor error calculation unit, 105 Correction unit, 106, 106a, 106b Output unit, 2, 2a Detection sensor, 21 Radio wave sensor, 22 Camera, 23 Fixed part, 24 Board, 25 Control unit, 3 Monitoring device, 31 Occupant detection device, 32 Physical size determination device, 4, 4a, 4b Information processing system, 1001 Processing circuit, 1002 Input interface device, 1003 Output interface device, 1004 Processor, 1005 Memory.

Claims

1. an image acquisition unit that acquires an image captured by the camera from a detection sensor that includes a camera that is provided in the vehicle interior and captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor that generates moving object distribution information that three-dimensionally represents a distribution of an area in the vehicle interior where a moving object is present, based on radio waves that are emitted toward the target area in the vehicle interior and reflected by an object in the vehicle interior; a camera error detection unit that detects an installation error of the camera based on the captured image acquired by the image acquisition unit; a radio wave sensor error calculation unit that calculates the installation error of the radio wave sensor based on the installation error of the camera detected by the camera error detection unit; an output unit that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor into moving object distribution information that takes into account the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit; An information processing device comprising:

2. a moving object distribution information acquisition unit that acquires the moving object distribution information from the detection sensor; a correction unit that corrects the moving object distribution information acquired by the moving object distribution information acquisition unit based on the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit, The output unit outputs the moving object distribution information corrected by the correction unit as the error consideration information.

2. The information processing apparatus according to claim 1, wherein:

3. a moving object distribution information acquisition unit that acquires the moving object distribution information from the detection sensor, The output unit outputs, as the error consideration information, the moving object distribution information acquired by the moving object distribution information acquisition unit and sensor installation error information indicating the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit.

2. The information processing apparatus according to claim 1, wherein:

4. The output unit outputs, as the error consideration information, instruction information for instructing the detection sensor to change the direction of radiation of the radio wave by the radio wave sensor.

2. The information processing apparatus according to claim 1, wherein:

5. the installation error of the camera includes an error in the installation position of the camera or an error in the orientation of the camera; The installation error of the radio wave sensor includes an error in the installation position of the radio wave sensor or an error in the direction of radiation of the radio waves from the radio wave sensor.

2. The information processing apparatus according to claim 1, wherein:

6. Located inside the vehicle, A fixing part; a camera that is provided on the fixed component and captures an image of a target area in the vehicle interior where at least an occupant may be present; a radio wave sensor that is provided on the fixed part and that generates moving object distribution information that three-dimensionally represents a distribution of areas where moving objects exist within the vehicle cabin based on radio waves that are emitted toward the target area within the vehicle cabin and reflected by objects within the vehicle cabin; an image acquisition unit that acquires an image captured by the camera; a camera error detection unit that detects an installation error of the camera based on the captured image acquired by the image acquisition unit; a radio wave sensor error calculation unit that calculates the installation error of the radio wave sensor based on the installation error of the camera detected by the camera error detection unit; an output unit that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor into moving object distribution information that takes into account the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit, When the radio wave sensor is installed in the vehicle cabin, at least two of the three-dimensional coordinate axes of the camera, where the x-axis and y-axis respectively refer to axes on the plane of the image sensor of the camera and the z-axis refers to an axis in the normal direction to the image sensor, and the three-dimensional coordinate axes of the radio wave sensor, where the x-axis and y-axis respectively refer to axes on the surface of a board on which an antenna is arranged and the z-axis refers to an axis in the normal direction to the board surface, coincide with each other. A detection sensor characterized by:

7. Located inside the vehicle, a camera that captures an image of a target area in the vehicle interior where at least an occupant may be present; a radio wave sensor that generates moving object distribution information that three-dimensionally represents a distribution of an area where moving objects exist within the vehicle cabin based on a wave of radio waves that are radiated toward the target area within the vehicle cabin and reflected by an object within the vehicle cabin, the radio wave sensor having a transmitting antenna that radiates the radio waves and a receiving antenna that receives the reflected wave of the radio waves radiated from the transmitting antenna; a substrate on which the camera, the transmitting antenna, and the receiving antenna are provided on a common surface; an image acquisition unit that acquires an image captured by the camera; a camera error detection unit that detects an installation error of the camera based on the captured image acquired by the image acquisition unit; a radio wave sensor error calculation unit that calculates the installation error of the radio wave sensor based on the installation error of the camera detected by the camera error detection unit; an output unit that outputs error consideration information for converting the moving object distribution information generated by the radio wave sensor into moving object distribution information that takes into account the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit; A detection sensor comprising:

8. The camera is disposed between the transmitting antenna and the receiving antenna on a common surface of the substrate.

8. The detection sensor according to claim 7.

9. a moving object distribution information acquisition unit that acquires the moving object distribution information; a correction unit that corrects the moving object distribution information acquired by the moving object distribution information acquisition unit based on the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit, The output unit outputs the moving object distribution information corrected by the correction unit as the error consideration information. The detection sensor according to any one of claims 6 to 8.

10. a moving object distribution information acquisition unit that acquires the moving object distribution information, The output unit outputs, as the error consideration information, the moving object distribution information acquired by the moving object distribution information acquisition unit and sensor installation error information indicating the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit. The detection sensor according to any one of claims 6 to 8.

11. a control unit that changes the direction of radiation of the radio wave from the radio wave sensor; The output unit outputs, to the control unit, instruction information for instructing the control unit to change the direction of radiation of the radio wave from the radio wave sensor, as the error consideration information. The detection sensor according to any one of claims 6 to 8.

12. the installation error of the camera includes an error in the installation position of the camera or an error in the orientation of the camera; The installation error of the radio wave sensor includes an error in the installation position of the radio wave sensor or an error in the direction of radiation of the radio waves from the radio wave sensor. The detection sensor according to any one of claims 6 to 8.

13. An information processing device according to any one of claims 1 to 5; The detection sensor is provided in the vehicle interior and includes a fixed part, the camera provided on the fixed part, and the radio wave sensor provided on the fixed part, wherein, when the detection sensor is installed in the vehicle interior, at least two of the three-dimensional coordinate axes of the camera, where the x-axis and y-axis respectively represent axes on the plane of an image pickup element of the camera and the z-axis represents an axis in the normal direction to the image pickup element, and the three-dimensional coordinate axes of the radio wave sensor, where the x-axis and y-axis respectively represent axes on the surface of a board on which an antenna is arranged and the z-axis represents an axis in the normal direction to the board surface, coincide with each other. An information processing system comprising:

14. An information processing device according to any one of claims 1 to 5; the radio wave sensor is provided in the vehicle interior and has the camera, a transmitting antenna that radiates the radio wave, and a receiving antenna that receives the reflected wave of the radio wave radiated from the transmitting antenna; and the detection sensor has a substrate on which the camera, the transmitting antenna, and the receiving antenna are provided on a common surface. An information processing system comprising:

15. The camera is disposed between the transmitting antenna and the receiving antenna on a common surface of the substrate.

15. The information processing system according to claim 14.

16. In the information processing device, the output unit outputs instruction information to the detection sensor to instruct the radio wave sensor to change a direction of radiation of the radio waves; The detection sensor includes a control unit that changes the direction of the radio wave emitted by the radio wave sensor based on the instruction information output from the output unit.

15. The information processing system according to claim 14.

17. an image acquisition unit provided in the vehicle interior, the image acquisition unit acquiring an image captured by the camera from a detection sensor having a camera that captures an image of a target area in the vehicle interior where at least an occupant may be present, and a radio wave sensor that generates moving object distribution information that three-dimensionally represents a distribution of an area where moving objects exist in the vehicle interior based on a radio wave that is emitted toward the target area in the vehicle interior and reflected by an object in the vehicle interior; a step in which a camera error detection unit detects an installation error of the camera based on the captured image acquired by the image acquisition unit; a radio wave sensor error calculation unit calculating the installation error of the radio wave sensor based on the installation error of the camera detected by the camera error detection unit; an output unit outputting error consideration information for converting the moving object distribution information generated by the radio wave sensor into moving object distribution information in which the installation error of the radio wave sensor calculated by the radio wave sensor error calculation unit is taken into consideration; An information processing method comprising:

Images