Occupant detection device and occupant detection method

Abstract

Occupant detection device (100) comprising an image acquisition unit (110) for acquiring an image produced by an imaging device, an area identification unit (120) for outputting image interior information by which, in the image acquired by the image acquisition unit, an area of ​​vehicle equipment, an area of ​​a living being, and an area of ​​an object on a seat are distinguished, an area assessment unit (130) for assessing, by means of the image interior information output by the area identification unit, with respect to an obscured area created in the image by the vehicle equipment, an area of ​​attention in which a living being may be located, for each living being concerned, a radio sensor data acquisition unit (140) for acquiring, by means of a result acquired by a radio sensor, sequentially, radio intensity data,which show the distribution of the radio intensity of reflection waves, a detection validity range / threshold optimization unit (150) for selecting, from the radio intensity data acquired by the radio sensor data acquisition unit, radio intensity data in the attention range assessed by the range estimation unit, a living being detection unit (160) for detecting, by means of the radio intensity data selected by the detection validity range / threshold optimization unit, an attribute of a living being, and an occupant information output unit (170) for generating, by means of the detection result of the living being detection unit, occupant information representing the occupants of the vehicle, and for outputting the occupant information to an occupant information processing device.

Description

Technical area

[0001] The technology of the present disclosure relates to an occupant detection technology. Background technology

[0002] There is an occupant detection technology that detects an occupant using images captured by an imaging device provided in a vehicle interior. There is also an occupant detection technology that seeks to detect attributes of an occupant, such as an infant / toddler (also referred to as "attributes of an occupant" or "attributes of a living being").

[0003] In cited document 1, an occupant detection device is disclosed in which the sitting / non-seating of an infant / toddler on a child seat equipped in a rear seat is detected.

[0004] Specifically, the occupant detection device comprises a first sensor and a second sensor. The first sensor is an image sensor (corresponding to the above "imaging device") that images the interior of the passenger compartment of a vehicle and is arranged in the front region of the passenger compartment in the vehicle length direction. The second sensor is provided so that a rear seat occupant can be detected. The second sensor is arranged on the ceiling of the passenger compartment behind the center of the seat surface of the rear seats in the vehicle length direction.

[0005] In the occupant detection device of cited document 1, it is determined whether a rear seat is equipped with a child seat, and if it is determined that it is equipped with a child seat, the sitting / not sitting of an infant / toddler on the child seat is detected by a detection from above by the second sensor related to the child seat. Prior art documentsPatent documents

[0006] Patent Document 1: Patent Publication No. JP 2019-123354 A Overview of the inventionProblem to be solved by the invention

[0007] However, it can happen that a passenger's seating position or seating condition is not constant. For example, an infant / toddler or a school-age child may sit in different postures in a location other than the child seat.

[0008] The problem with occupant detection technology is that in such a case, there is a tendency for misdetection of the occupant's attributes, and the precision in detecting the occupant's attributes is low.

[0009] The occupant detection device specified in cited document 1 can detect whether an occupant sitting in the child seat is an infant / toddler when equipped with a child seat, but this cannot solve the above problem.

[0010] The present disclosure is intended to solve this problem and to improve the precision in detecting the attributes of an occupant. Means of solving the task

[0011] The occupant detection device of the present invention comprises an image acquisition unit for acquiring an image imaged by an imaging device, an area identification unit for outputting image interior information by which an area of ​​a vehicle device, an area of ​​a living being, and an area of ​​an object on a seat are distinguished in the image acquired by the image acquisition unit, an area estimation unit for estimating, using the image interior information output by the area identification unit, an attention area in which a living being may be located for each living being concerned, with respect to an obscured area created in the image by the vehicle device, a radio sensor data acquisition unit for acquiring, using a result detected by a radio sensor, in time sequence, radio intensity data,which show the distribution of the radio intensity of reflected waves, a detection validity range / threshold optimization unit for selecting, from the radio intensity data acquired by the radio sensor data acquisition unit, radio intensity data in the attention range estimated by the area estimation unit, a living being detection unit for detecting, using the radio intensity data selected by the detection validity range / threshold optimization unit, an attribute of a living being, and an occupant information output unit for generating, using the detection result of the living being detection unit, occupant information representing the occupants of the vehicle and outputting the occupant information to an occupant information processing device. Effects of the invention

[0012] According to the present disclosure, the precision with which attributes of an occupant are detected can be improved. Brief explanation of the drawings [ Fig. 1] Fig. 1 is a diagram showing a structural example of an occupant detection system including an occupant detection device according to the present disclosure. [ Fig. 2] Fig. 2 is a diagram showing an arrangement example of an occupant detection system including an occupant detection device according to the present disclosure. [ Fig. 3] Fig. 3 is a flowchart showing an example of processing of the occupant detection device according to the present disclosure. [ Fig. 4] Fig. 4 is a view for explaining an example of occupant detection by the occupant detection device according to the first embodiment, which is a perspective view of a vehicle interior. [ Fig. 5] Fig. 5 is a view for explaining an example of occupant detection by the occupant detection device according to the first embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior. [ Fig. 6] Fig. 6 is a view for explaining an example of occupant detection by the occupant detection apparatus according to the first embodiment, which is a view for explaining area estimation processing. [ Fig. 7] Fig. 7 is a view for explaining an example of occupant detection by the occupant detection device according to the first embodiment, which is a view for explaining processing in which a living being is detected by a radio intensity by a radio sensor. [ Fig. 8] Fig. 8 is a view for explaining an example of occupant detection by the occupant detection device according to the first embodiment, which is a view showing a state where there is a possibility that an occupant has been left behind. [ Fig. 9] Fig. 9 is a view for explaining an example of occupant detection by the occupant detection device according to a second embodiment, which is a perspective view of a vehicle interior. [ Fig. 10] Fig. 10 is a view for explaining an example of occupant detection by the occupant detection device according to the second embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior. [ Fig. 11] Fig. 11 is a view for explaining an example of occupant detection by the occupant detection device according to the second embodiment, which is a view for explaining area estimation processing. [ Fig. 12] Fig. 12 is a view for explaining an example of occupant detection by the occupant detection device according to the second embodiment, which is a view for explaining processing in which a living being is detected by a radio intensity by a radio sensor. [ Fig. 13] Fig. 13 is a view for explaining an example of occupant detection by the occupant detection device according to the second embodiment, which is a view showing a state where there is a possibility that an occupant has been left behind. [ Fig. 14] Fig. 14 is a view for explaining an example of occupant detection by the occupant detection device according to a third embodiment, which is a perspective view of a vehicle interior. [ Fig. 15] Fig. 15 is a view for explaining an example of occupant detection by the occupant detection device according to the third embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior. [ Fig. 16] Fig. 16 is a view for explaining an example of occupant detection by the occupant detection device according to the third embodiment, which is a view for explaining area estimation processing. [ Fig. 17] Fig. 17 is a view for explaining an example of occupant detection by the occupant detection device according to the third embodiment, which is a view for explaining processing in which a living being is detected by a radio intensity by a radio sensor. [ Fig. 18] Fig. 18 is a diagram explaining another example of occupant detection by the occupant detection device according to the present disclosure. [ Fig. 19] Fig. 19 is a view showing a first example of a hardware configuration for realizing the functions of the occupant detection apparatus according to the present disclosure. [ Fig. 20] Fig. 20 is a view showing a second example of a hardware configuration for realizing the functions of the occupant detection apparatus according to the present disclosure. Forms for carrying out the invention

[0013] In the following, in order to explain the present disclosure in detail, forms for carrying out the present disclosure will be explained with reference to the accompanying drawings. First embodiment

[0014] Fig. 1 is a diagram showing a structural example of an occupant detection system 1 including an occupant detection device 100 according to the present disclosure.

[0015] The occupant detection system 1 is a system by which occupants of a mobile body are detected.

[0016] The occupant detection system 1 monitors the state of the interior space in the mobile body (mobile body interior) by means of the detection results of various sensors incorporated in the mobile body.

[0017] The occupant detection system 1 detects an occupant of the mobile body, e.g., by detecting the presence / absence of a living being, the number of living beings, or attributes of a living being in the mobile body interior.

[0018] A living being includes living beings other than a human, such as an animal, for example a human's pet.

[0019] The following explains the occupant detection system 1, assuming that the mobile body is a vehicle. In the present disclosure, the term "vehicle" may be replaced by "mobile body" within a technically reasonable range.

[0020] The Fig. The occupant detection system 1 shown in Fig. 1 has a structure including an occupant detection device 100, a camera 200 (imaging device), a wireless sensor 210, and an occupant information processing device 300.

[0021] The occupant detection device 100 is communicatively connected to the camera 200, the wireless sensor 210, and the occupant information processing device 300. The occupant detection device 100 and the occupant information processing device 300 are connected, for example, via a CAN (Controller Area Network).

[0022] The occupant detection device 100 detects the presence / absence of a living being in the vehicle interior of a vehicle using images and wireless sensor data.

[0023] The occupant detection device 100 detects the number of living beings in the vehicle interior of a vehicle using images and wireless sensor data. The occupant detection device 100 detects attributes of a living being in the vehicle interior of a vehicle using images and wireless sensor data. The occupant detection device 100 generates occupant information based on either the presence / absence of a living being, the number of living beings, or attributes of living beings.

[0024] The occupant detection device 100 uses images to estimate at least one attention zone in which a living being could be located in an area obscured by vehicle equipment. The occupant detection device 100 detects a living being located in an attention zone in the vicinity of a vehicle equipment using a radio intensity represented by the radio sensor data.

[0025] The internal structure of the occupant detection device 100 is described below.

[0026] The camera 200 is the imaging device in the present disclosure. The camera 200 outputs imaged images (image images).

[0027] The camera 200 is mounted to image the vehicle interior. A more preferred mounting location for the camera 200 is, for example, the front or upper front area of ​​the vehicle.

[0028] The radio sensor 210 is a sensor that uses radio waves and can detect a living being in the rear seat through the front seats.

[0029] The radio sensor 210 transmits radio waves, receives reflected waves and outputs a radio signal.

[0030] The wireless sensor 210 is mounted such that it emits radio waves directed toward the vehicle interior. In a more preferred mounting of the wireless sensor 210, the wireless sensor 210 is mounted, for example, in the vehicle interior in the front or upper front area of ​​the vehicle. Such a mounting allows the wireless sensor 210 to transmit radio waves into an area including an attention zone within the vehicle interior and to receive reflected waves from the area including the attention zone.

[0031] More preferably, the mounting position of the camera 200 and the mounting position of the wireless sensor 210 are approximately the same. This eliminates the need for processing to align the coordinates of the vehicle interior included in images captured by the camera 200 with the coordinates of the wireless intensity represented in the wireless sensor data, thus reducing the processing load of the occupant detection device 100.

[0032] The occupant information processing device 300 acquires occupant information and performs processing using the occupant information.

[0033] The occupant information processing device 300 is, for example, a notification device that generates notification information based on the presence / absence of a living being, the number of living beings, or attributes of living beings that may be included in the occupant information, and outputs the notification information via an image output device (an unillustrated image output device), a sound output device (an unillustrated sound output device), a terminal, a portable terminal, or a smartphone.

[0034] The occupant information processing device 300 is not limited to the above notification device as long as processing is performed using occupant information. For example, the occupant information processing device 300 may also be a vehicle control device that performs vehicle control using occupant information.

[0035] The occupant information processing device 300 may also be configured to include the occupant detection device 100.

[0036] An example of the internal structure of the occupant detection device 100 will be explained. The structure of the occupant detection device 100 includes an image acquisition unit 110, an area identification unit 120, a dictionary data unit 125, an area estimation unit 130, a wireless sensor data acquisition unit 140, a detection validity range / threshold optimization unit 150, a living being detection unit 160, and an occupant information output unit 170.

[0037] The image acquisition unit 110 acquires an image imaged by an imaging device.

[0038] In a more preferable use state of the occupant detection device 100, the image acquisition unit 110 acquires an image of the vehicle interior imaged by an imaging device with which the vehicle interior is equipped.

[0039] Specifically, when the image acquisition unit 110 acquires image signals from an imaging device, it performs processing such as separation into IR images and RGB images, noise removal, and distortion correction processing to acquire an image.

[0040] The area identifying unit 120 outputs image interior area information by which areas of vehicle equipment, areas of living beings, and areas of objects on a seat are distinguished in the images acquired by the image acquiring unit 110.

[0041] "Vehicle fixtures" are devices that the vehicle is generally equipped with at the time of its manufacture, such as the front or rear seats. "Vehicle fixtures" also include devices inside the vehicle that obscure a space between the camera and the seat of a subject. An "object" in "objects on a seat" refers to an item installed after the vehicle was manufactured, such as a child seat (an infant / toddler seat) or simply an object resting on a seat.

[0042] Specifically, the area identifying unit 120 identifies, in the images acquired by the image acquiring unit 110, an area for each vehicle device, an area for each living being, and an area for each object on a seat, and outputs image interior area information for distinguishing the identified respective areas.

[0043] More specifically, the area identification unit 120 first acquires images output from the image acquisition unit 110. Next, the area identification unit 120 performs image recognition processing using the images acquired by the image acquisition unit 110. The accuracy of the area identification can be increased by the area identification unit 120 performing image recognition processing with reference to, for example, dictionary data 125 (dictionary data 125 described below). The image recognition itself can be realized using existing technology, so detailed explanations are omitted. The area identification unit 120 identifies, in the images, an area for each vehicle device, an area for each living being, and an area for each object on a seat.The area identification unit 120 generates image interior area information for distinguishing the identified areas for each vehicle device, the areas for each living being, and the areas for each object on a seat. The form of the image interior area information is not limited as long as it is information that can distinguish the areas for each vehicle device, the areas for each living being, and the areas for each object on a seat in the images. The area identification unit 120 outputs the generated image interior area information.

[0044] The areas of objects on a seat identified by the area identification unit 120 include sub-areas of objects. The areas of objects on a seat include, for example, sub-areas of a child seat.

[0045] The areas of living beings identified by the area identification unit 120 include sub-areas of living beings. The sub-areas of living beings are, for example, areas such as the head, neck, chest, abdomen, hips, thighs, or legs of a living being.

[0046] The dictionary data 125 is learning data learned for image recognition. The dictionary data 125 can be appropriately referenced in the processing of the region identification unit 120, the region estimation unit 130, and the living being detection unit 160.

[0047] The area estimation unit 130 estimates, using the image interior area information output from the area identification unit 120, an attention area in which the living being may be located for each living being concerned, with respect to a hidden area created in an image by vehicle devices.

[0048] Specifically, the area estimation unit 130 extracts the obscured area using the image interior area information. An obscured area is an area that obscures the rear seats among the areas of vehicle equipment in an image. The area estimation unit 130 extracts living being areas and areas of objects on a seat in an image using the image interior area information. The area estimation unit 130 estimates an attention area in the obscured area where a living being may be located using the extracted living being areas and areas of objects on a seat.

[0049] More specifically, the area estimation unit 130 estimates an attention area that is smaller than the occupant area. This allows the occupant detection device 100 to more easily detect a living being when a living being whose entire body is occupied is located in the occupant area. Furthermore, based on the size of the attention area, the occupant detection device 100 can more easily detect attributes of a living being with a relatively small stature compared to an adult, such as an infant, a toddler, and a schoolchild.

[0050] More specifically, when the area of ​​objects on a seat identified by the area identification unit 120 is a partial area of ​​a child seat, the area estimation unit 130 can estimate an attention area by estimating the entire area of ​​a child seat based on the area of ​​objects on a seat identified by the area identification unit 120. This makes it easier for the occupant detection device 100 to detect attributes of a living being such as an infant or toddler as an occupant sitting in a child seat.

[0051] More specifically, the area estimation unit 130 further estimates an attention area in the occupant area adjacent to a living being represented by the image interior area information. This allows the occupant detection device 100 to more easily detect attributes of a living being with a relatively small stature compared to an adult, such as an infant, a toddler, and a schoolchild.

[0052] More specifically, when the area of ​​living beings represented by the image interior area information is a partial area in contact with the seat surface of a rear seat, the area estimation unit 130 can further estimate an attention area using the partial area by estimating an area where a living being may be located that extends across the partial area and the occupant area. This makes it easier for the occupant detection device 100 to detect attributes of a living being with a relatively small stature compared to an adult, such as an infant, a toddler, and a schoolchild, who is an occupant lying on a rear seat, for example.

[0053] Furthermore, the area estimation unit 130 also determines living being detection areas in which a living being is located using the image interior area information output from the area identification unit 120. This improves the accuracy of detecting the number of living beings in the occupant detection device 100.

[0054] The area estimation unit 130 outputs attention area information representing an attention area or living being detection area information representing a living being detection area according to the estimated result.

[0055] The radio sensor data acquisition unit 140 cyclically acquires radio intensity sensors showing the distribution of radio intensity of reflection waves using a result detected by the radio sensor 210.

[0056] Specifically, the wireless sensor data acquisition unit 140 first acquires a wireless signal as a result detected by the wireless sensor 210 regarding an area including an attention zone. The wireless sensor data acquisition unit 140 then performs basic wireless signal processing such as FFT (Fast Fourier Transform) on the wireless signal and generates wireless intensity data showing the distribution of the wireless intensity of the reflected waves. The wireless sensor data acquisition unit 140 cyclically generates wireless intensity data and outputs the generated wireless intensity data. The wireless intensity data is data that can express wireless intensity in the form of point group data such as a point cloud.

[0057] The detection validity range / threshold optimization unit 150 extracts radio intensity data usable for detection.

[0058] The detection validity range / threshold optimization unit 150 acquires radio intensity data from the radio sensor data acquisition unit 140 and selects, from the radio intensity data acquired by the radio sensor data acquisition unit, radio intensity data in an attention range estimated by the range estimation unit.

[0059] Specifically, the detection validity range / threshold optimization unit 150 acquires radio intensity data from the radio sensor data acquisition unit 140. The detection validity range / threshold optimization unit 150 acquires attention range information from the area estimation unit 130.

[0060] The detection validity range / threshold optimization unit 150 sets an attention range within a detection validity range using the attention range information, and selects radio intensity data within the attention range from the radio intensity data acquired by the radio sensor data acquisition unit. The detection validity range / threshold optimization unit 150 thereby selects a radio intensity sensor within an optimized detection validity range.

[0061] The detection validity range / threshold optimization unit 150 also selects, from the radio intensity data in the attention area estimated by the area estimation unit, radio intensity data that exceeds the relevant threshold using a threshold value corresponding to the area represented by the image interior area information. For example, if the image interior area information represents a child safety seat area, the detection validity range / threshold optimization unit 150 uses a threshold value that allows radio intensity data in a child safety seat area to be selected to have a weaker electrical power than the electrical power upon direct reflection from a living being. The detection validity range / threshold optimization unit 150 thereby selects the radio intensity data using an optimal threshold value.

[0062] The detection validity range / threshold optimization unit 150 can also be configured to adjust the radio intensity data of the attention area. In this case, the detection validity range / threshold optimization unit 150 performs signal processing to correct the radio intensity data of the attention area according to a vehicle device or an object on a seat located in the obscured area. Specifically, the threshold optimization unit 150 performs, for example, a calculation operation in which the radio intensity value represented by the radio intensity data is calculated. Furthermore, the threshold optimization unit 150 performs, for example, filter processing of the radio intensity value represented by the radio intensity data, taking into account the change over time.The detection validity range / threshold optimization unit 150 selects, with respect to the radio intensity data after adjustment by the above threshold, the radio intensity data exceeding the threshold in question and outputs it.

[0063] Were as described in the following Fig. 18 detection target areas are preset, the detection validity area / threshold optimization unit 150, when acquiring the attention area information from the area estimation unit 130, newly sets an attention area as the detection validity area instead of the detection target areas as stated above.

[0064] The detection validity range / threshold optimization unit 150 may also further adjust the radio intensity by a command to the radio sensor 210 so that the sensitivity of the radio sensor 210 is increased based on the radio intensity represented by the radio intensity data of the attention area.

[0065] The living being detection unit 160 detects attributes of a living being using the image inner region information and the radio intensity data selected by the detection validity region / threshold optimization unit 150.

[0066] Specifically, the living being detection unit 160 detects attributes of a living being using the areas for each living being in the image and the radio intensity data for each attention area selected by the detection validity area / threshold optimization unit 150.

[0067] More specifically, the living being detection unit 160 uses the radio intensity data acquired sequentially for each area to determine the size and position of a reflection object in a detection target area by summing the distribution and cumulative intensity of the radio intensity over time and observing the distribution of the radio intensity over time. The living being detection unit 160 uses the result of determining the size and position of the reflection object in the detection target area to determine the number of living beings and attributes of living beings. The living being detection unit 160 outputs living being information including the number of living beings and attributes of living beings. Furthermore, if the area estimation unit 130 determines a living being detection area in which a living being is located, the living being detection unit 160 further uses the living being detection area to determine attributes of living beings.

[0068] Furthermore, if the area estimation unit 130 has determined a living being detection area in which a living being is located, the living being detection unit 160 further detects the number of living beings in the vehicle interior using the living being detection area, attention area, and radio intensity.

[0069] The occupant information output unit 170 generates occupant information representing an occupant who has boarded the vehicle using the detection result of the living being detection unit 160, and outputs the occupant information to the occupant information processing device 300. Specifically, the occupant information output unit 170 acquires living being information representing the detection result of the living being detection unit 160 and generates occupant information including the presence / absence of an occupant, the number of occupants, and occupant attributes. The occupant information may include, for example,also include information in the form of a flag indicating that an occupant is exclusively an infant / toddler, a flag indicating that an occupant is exclusively an infant / toddler or a school-age child, and a flag indicating that there is a possibility of being left behind. The occupant information output unit 170 outputs the relevant occupant information to the occupant information processing device 300.

[0070] Fig. 2 is a diagram showing an arrangement example of the occupant detection system 1 including the occupant detection device 100 according to the present disclosure.

[0071] Fig. 2 expresses a state of a vehicle 400 equipped with the occupant detection system 1 as seen from a cut-out side surface.

[0072] In the Fig. For example, in the vehicle 400 shown in Fig. 2, the occupant detection device 100 and the occupant information processing device 300 are arranged at the front of a vehicle interior 410.

[0073] The camera 200 is mounted, for example, in the vehicle interior 410 in the front or upper front area of ​​the vehicle.

[0074] The radio sensor 210 is mounted, for example, in the vehicle interior in the front or upper front area of ​​the vehicle.

[0075] The mounting position of the camera 200 and the mounting position of the wireless sensor 210 are approximately at the same location.

[0076] In an image captured by camera 200, front seats 411, as vehicle equipment, become obscuring objects, creating a blind spot 415 on the side of rear seats 412.

[0077] By the occupant detection device 100 according to the present disclosure, areas of occupants in an image captured by the camera 200 can be detected by means of the detection result of the wireless sensor 210, even if an occupant such as an infant, a toddler, or a schoolchild is located in the blind spot 415.

[0078] Fig. 3 is a flowchart showing an example of processing of the occupant detection device 100 according to the present disclosure.

[0079] The occupant detection device 100 begins with the Fig. 3, for example, when an imaging device starts imaging. Or the occupant detection device 100 starts the Fig. 3, for example, when the vehicle's engine has been switched off or when the driver has gotten out of the vehicle.

[0080] The occupant detection device 100 begins with a Fig. 3 shown loop processing (step ST100).

[0081] The occupant detection device 100 executes image acquisition processing (step ST101).

[0082] Specifically, in the occupant detection device 100, when the image acquisition unit 110 acquires image signals from an imaging device, it performs processing such as separation into IR images and RGB images, noise removal, and distortion correction to acquire an image.

[0083] The image acquisition unit 110 outputs an acquired image to the area identification unit 120.

[0084] The occupant detection device 100 executes area identification processing (step ST102).

[0085] Specifically, in the occupant detection device 100, the area identification unit 120 first acquires images output from the image acquisition unit 110. Next, the area identification unit 120 performs image recognition processing using the images acquired by the image acquisition unit 110. The area identification unit 120 identifies, in the images, an area for each vehicle device, an area for each living being, and an area for each object on a seat. The area identification unit 120 generates image interior area information for distinguishing the identified areas for each vehicle device, areas for each living being, and areas for each object on a seat. The area identification unit 120 outputs the generated image interior area information.

[0086] The occupant detection device 100 executes area estimation processing (step ST103).

[0087] Specifically, in the occupant detection device 100, the area estimation unit 130 first acquires image interior area information from the area identification unit 120. The area estimation unit 130 uses the image interior area information to extract an obscured area, which is an area obscuring the rear seats among the areas of vehicle equipment in the image. Furthermore, the area estimation unit 130 uses the image interior area information to extract living being areas and areas of objects on a seat in the image. The area estimation unit 130 estimates an attention area where a living being may be located in the obscured area using the extracted living being areas and areas of objects on a seat.

[0088] The area estimation unit 130 estimates, for example, the area of ​​a living being that may be obscured by the obscured area using the position of a partial area of ​​a living being, and estimates an attention area that includes the estimated area.

[0089] Furthermore, the area estimation unit 130 estimates the entire area of ​​the child seat, for example, using a partial area of ​​a child seat, and estimates an attention area that includes the estimated area.

[0090] Furthermore, the area estimation unit 130 estimates, for example, by means of a partial area of ​​a living being, the total area of ​​a living being that may extend over the partial area and the hidden area, and estimates an attention area that includes the estimated area.

[0091] The occupant detection device 100 executes wireless sensor data acquisition processing (step ST104).

[0092] Specifically, in the occupant detection device 100, the wireless sensor data acquisition unit 140 first acquires a wireless signal as a result detected by the wireless sensor 210 regarding an area including an attention area.

[0093] The occupant detection device 100 performs signal processing (step ST105). Specifically, in the occupant detection device 100, the wireless sensor data acquisition unit 140 performs basic wireless signal processing on the wireless signal and generates wireless intensity data indicating the distribution of the wireless intensity of reflected waves. The wireless sensor data acquisition unit 140 cyclically generates wireless intensity data and outputs the generated wireless intensity data.

[0094] The occupant detection device 100 performs detection validity range / threshold optimization processing and selects radio intensity data valid for detection (step ST106).

[0095] Specifically, in the occupant detection device 100, the detection validity range / threshold optimization unit 150 first acquires radio intensity data from the radio sensor data acquisition unit 140. Further, the detection validity range / threshold optimization unit 150 acquires attention area information from the area estimation unit 130.

[0096] Next, the detection validity range / threshold optimization unit 150 sets an attention range within a detection validity range using the attention range information, and selects radio intensity data within the attention range from the radio intensity data acquired by the radio sensor data acquisition unit. The detection validity range / threshold optimization unit 150 thereby selects a radio intensity sensor within an optimized detection validity range.

[0097] Next, the detection validity range / threshold optimization unit 150 further selects, from the radio intensity data in the attention area estimated by the area estimation unit, radio intensity data exceeding the relevant threshold using a threshold corresponding to the area represented by the image interior area information. Thus, the detection validity range / threshold optimization unit 150 selects the radio intensity data using an optimized threshold.

[0098] In this case, the detection validity range / threshold optimization unit 150 can also be configured to adjust the radio intensity data of the attention area. In this case, the detection validity range / threshold optimization unit 150 performs signal processing, which corrects the radio intensity data of the attention area according to a vehicle device or an object on a seat located in the obscured area. Specifically, the detection validity range / threshold optimization unit 150 performs, for example, a calculation operation in which the radio intensity value represented by the radio intensity data is calculated. Furthermore, the threshold optimization unit 150 performs, for example, filter processing of the radio intensity value represented by the radio intensity data, taking into account the change over time.The detection validity range / threshold optimization unit 150 selects, with respect to the radio intensity data after adjustment by the above threshold, the radio intensity data exceeding the threshold in question and outputs it.

[0099] The detection validity range / threshold optimization unit 150 may further adjust the radio intensity by a command to the radio sensor 210 so that the sensitivity of the radio sensor 210 is increased based on the radio intensity represented by the radio intensity data of the attention area.

[0100] The occupant detection device 100 executes living being detection processing (step ST107).

[0101] Specifically, in the occupant detection device 100, the living being detection unit 160 acquires radio intensity data over time selected by the detection validity range / threshold optimization unit. Using the radio intensity data, it detects the size and position of a reflection object in the detection target area by summing the distribution and cumulative intensity of the radio intensity over time and observing the distribution of the radio intensity over time. The living being detection unit 160 detects the number of living beings and attributes of living beings using the detection result of the size and position of the reflection object in the detection target area. The living being detection unit 160 outputs living being information including the number of living beings and attributes of living beings to the occupant information output unit 170.

[0102] The occupant detection device 100 executes occupant information output processing (step ST108).

[0103] Specifically, in the occupant detection device 100, the occupant information output unit 170 acquires living being information representing the detection result of the living being detection unit 160. The occupant information output unit 170 generates occupant information including the presence / absence of an occupant, the number of occupants, and occupant attributes using the living being information. The occupant information output unit 170 outputs the relevant occupant information to the occupant information processing device 300.

[0104] The occupant detection device 100 repeats the processing from step ST101 after executing the occupant information output processing in step ST108.

[0105] The occupant detection device 100 terminates the Fig. 3, e.g., when the imaging device finishes imaging.

[0106] A first example of occupant detection by the occupant detection device 100 will be explained.

[0107] Fig. 4 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the first embodiment, which is a perspective view of a vehicle interior.

[0108] Fig. 5 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the first embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior.

[0109] Fig. 6 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the first embodiment, which is a view for explaining area estimation processing.

[0110] Fig. 7 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the first embodiment, which is a view for explaining processing in which a living being is detected by the radio intensity by the radio sensor 210.

[0111] Fig. 8 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the first embodiment, which is a view showing a state where there is a possibility that an occupant has been left behind.

[0112] An arrow A in Fig. 4 shows a direction seen, for example, from the mounting position of the camera 200.

[0113] In the vehicle interior, a living being 420 sits in the direction of the rear seats 412 on the left side, and a child seat 430 is attached to the right side. A living being 440 with attributes of an infant or toddler is seated on the child seat 430. In this state, an image acquired by the image acquisition unit 110, as shown in Fig. 5, a passenger seat 411-2 of the front seats 411 becomes an obscuring object, creating a blind spot. In this case, the living being 440 (infant / toddler) sitting on the child seat 430 cannot be detected in the image captured by the camera 200 because it is obscured by the passenger seat 411-2 of the front seats 411 in a concealed area.

[0114] The area identification unit 120 identifies the areas of the driver's seat 411-1 and the passenger seat 411-1 of the front seats 411, and the area of ​​the rear seats 412, as areas of vehicle equipment. Furthermore, the area identification unit 120 identifies the area of ​​the living being 420. The area identification unit 120 further identifies a partial area of ​​the child seat 430 as an area of ​​an object on a seat.

[0115] The area estimation unit 130 estimates, as in Fig. 6, an attention area 455.

[0116] Specifically, the area estimation unit 130 estimates the total area of ​​the child seat 430 using the partial area of ​​the child seat 430, and estimates the attention area 455 in which the living being 440, ie an infant / toddler, could be located based on the total area of ​​the child seat 430.

[0117] Here, the area estimation unit 130 can also simultaneously estimate the entire area of ​​the living being 420 using the partial area of ​​the living being 420, and define a living being detection area 450 based on the entire area of ​​the living being 420.

[0118] By means of the processing by the radio sensor data acquisition unit 140 and the detection validity range / threshold optimization unit 150, Fig. 7 illustrated data acquired by the Living Being Detection Unit 160.

[0119] The living being detection unit 160 detects the living being 440 with attributes of an infant / toddler using the area of ​​the child seat 430 included in the image and the radio intensities 465 acquired in the attention area 455.

[0120] If done according to Fig. 4 to Fig. 7 shown state a transition to the one in Fig. 8 shown condition, or simply in the Fig. 8, in the occupant detection device 100, information that the occupants are exclusively an infant / toddler, information that the occupants are exclusively an infant / toddler or a school-age child, or information about the possibility of being left behind can be output by the occupant information output unit 170.

[0121] The structure of the occupant detection device according to the present disclosure includes an image acquisition unit that acquires an image imaged by an imaging device; an area identification unit that outputs image interior information for distinguishing areas of vehicle equipment, areas of living beings, and areas of objects on a seat in the image acquired by the image acquisition unit; an area estimation unit that estimates, for each living being concerned, an attention area in which a living being may be located, using the image interior information output by the area identification unit with respect to an obscured area created in the image by the vehicle equipment; a wireless sensor data acquisition unit that acquires wireless intensity data in a time-series manner using a result detected by a wireless sensor;which show the distribution of the radio intensity of reflected waves, a detection validity range / threshold optimization unit that selects radio intensity data in the attention range estimated by the area estimation unit from the radio intensity data acquired by the radio sensor data acquisition unit, a living being detection unit that detects attributes of a living being using the radio intensity data selected by the detection validity range / threshold optimization unit, and an occupant information output unit that generates occupant information representing the occupants of the vehicle using the detection result of the living being detection unit and outputs the occupant information to an occupant information processing device.

[0122] As a result, the present disclosure makes it possible to provide an occupant detection apparatus in which the precision in detecting attributes of an occupant is improved.

[0123] The structure of the occupant detection method of the present disclosure includes an image acquisition step in which an image acquired by an imaging device is acquired by an image acquisition unit; an area identification step in which an area identification unit outputs image interior information by which areas of vehicle devices, areas of living beings, and areas of objects on a seat are distinguished in the image acquired by the image acquisition unit; an area estimation step in which an attention area in which a living being may be located is estimated for each living being concerned by an area estimation unit using the image interior information output by the area identification unit with respect to an obscured area created in the image by the vehicle devices; a wireless sensor data acquisition step;in which a radio sensor data acquisition unit acquires radio intensity data showing the distribution of the radio intensity of reflected waves in time sequence using a result detected by a radio sensor; a detection validity range / threshold optimization step in which a detection validity range / threshold optimization unit selects radio intensity data in the attention range estimated by the area estimation unit from the radio intensity data acquired by the radio sensor data acquisition unit; a living being detection step in which attributes of a living being are detected by a living being detection unit using the radio intensity data selected by the detection validity range / threshold optimization unit; and an occupant information output step,in which an occupant information output unit generates occupant information representing the occupants of the vehicle using the detection result of the living being detection unit, and the occupant information is output to an occupant information processing device.

[0124] As a result, the present disclosure provides an occupant detection method in which the precision of detecting attributes of an occupant is improved.

[0125] The occupant detection device according to the present disclosure is of a configuration in which the detection validity range / threshold optimization unit further selects, from the radio intensity data in the attention range estimated by the range estimation unit, radio intensity data exceeding the threshold value by means of a threshold value corresponding to the range represented by the image inner range information.

[0126] As a result, the present disclosure is capable of extracting radio intensity data effective for detection, for example, considering a seat such as a child seat mounted in the vehicle interior as a vehicle device, and improving the precision in detecting attributes of a living being by the radio intensity data in the attention range.

[0127] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0128] The occupant detection device according to the present disclosure is configured such that the area estimation unit estimates the attention area smaller than the occupant obscuration area. This makes it easier to detect an occupant with a relatively small stature compared to an adult, such as an infant, a toddler, and a schoolchild.

[0129] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0130] The occupant detection device according to the present disclosure is of a configuration in which the area of ​​objects on a seat identified by the area identifying unit is a partial area of ​​a child seat, and the area estimating unit estimates the attention area by estimating the entire area of ​​the child seat using the area of ​​objects on a seat identified by the area identifying unit.

[0131] This makes it easier for the present disclosure to detect an occupant having a relatively small stature compared to an adult, such as an infant or toddler.

[0132] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0133] The occupant detection device according to the present disclosure is of a configuration in which the area estimation unit estimates the attention area by estimating an area in which a living being may be located extending across the partial area and the hidden area, using the partial area when the area of ​​living beings represented by the image inner area information is a partial area in contact with the seat surface of a rear seat.

[0134] As a result, the present disclosure makes it easier to detect, for example, an occupant lying on a rear seat.

[0135] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0136] The occupant detection device according to the present disclosure is a configuration in which the area estimation unit further sets a living being detection area in which a living being is located using image inside area information output from the area identifying unit, and the living being detection unit further detects attributes of living beings using the living being detection area.

[0137] This makes it possible in the present disclosure to detect an occupant obscured by an obscuring object and an occupant not obscured by an obscuring object.

[0138] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0139] The occupant detection device according to the present disclosure is a structure in which the living matter detection unit further detects the number of living matters in the vehicle interior using the living matter detection area, attention area, and radio intensity.

[0140] This has the effect of improving the precision of detecting the number of occupants in the present disclosure.

[0141] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0142] Second Embodiment In the second embodiment, a second example of occupant detection by the occupant detection device 100 is explained.

[0143] In the explanation of the second embodiment, an explanation of the structure explained in the first embodiment is appropriately omitted.

[0144] The structural example of the occupant detection device 100 according to the second embodiment is the same as the structural example of the occupant detection device 100 explained in the first embodiment, so a detailed explanation of the structural example will be omitted.

[0145] The second example of occupant detection by the occupant detection device 100 will be explained.

[0146] Fig. 9 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the second embodiment, which is a perspective view of a vehicle interior.

[0147] Fig. 10 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the second embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior.

[0148] Fig. 11 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the second embodiment, which is a view for explaining area estimation processing.

[0149] Fig. 12 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the second embodiment, which is a view for explaining processing in which a living being is detected by the radio intensity by the radio sensor 210.

[0150] Fig. 13 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the second embodiment, which is a view showing a state where there is a possibility that an occupant has been left behind.

[0151] An arrow B in Fig. 9 shows a direction seen, for example, from the perspective of the mounting position of the camera 200.

[0152] In the vehicle interior, a living being 520 sits in a section further toward the rear seats 412 than the rear seat on the right side. In this state, an image acquired by the image acquisition unit 110, as shown in Fig. 10, a passenger seat 411-2 of the front seats 411 becomes an obscuring object and a blind spot is created.

[0153] In the image captured by the camera 200, a living being 540 cannot be detected, although it includes a partial area of ​​the living being 520.

[0154] The area identification unit 120 identifies the areas of the driver's seat 411-1 and the passenger seat 411-2 of the front seats 411 and the area of ​​the rear seats 412 as areas of vehicle equipment. Furthermore, the area identification unit 120 identifies the area of ​​the living being 520.

[0155] The area assessment unit 130 estimates as in Fig. 11, an attention area 555.

[0156] Specifically, the area estimation unit 130 determines that there is a possibility of the living being 540 sitting further to the right side because the sitting position of the living being 520 is in a section of the right side that is more toward the center, and estimates the attention area 555 in which the living being 540 might be located.

[0157] Here, the area estimation unit 130 can also simultaneously estimate the entire area of ​​the living being 520 using the partial area of ​​the living being 520, and determine the living being detection area 550 based on the entire area of ​​the living being 520.

[0158] By means of the processing by the wireless sensor data acquisition unit 140 and the detection validity range / threshold optimization unit 150, the living being detection unit 160 acquires Fig. 12 illustrated data.

[0159] The living being detection unit 160 detects the living being 540 with attributes of an infant / toddler or schoolchild by means of the partial area of ​​the living being 520 encompassed in the image and by means of the radio intensities 465 acquired in the attention area 555.

[0160] If done according to Fig. 9 to Fig. 12 shown state a transition to the one in Fig. 13 shown condition, or simply in Fig. 13, in the occupant detection device 100, information that the occupant is exclusively an infant / toddler, information that the occupant is exclusively an infant / toddler or a school-age child, or information about the possibility of being left behind can be output by the occupant information output unit 170.

[0161] With regard to the second embodiment, areas of two living beings are identified in the image, which may also be the assessment of an attention area in which a living being (infant / toddler) sitting between the two living beings could be located.

[0162] The occupant detection device according to the present disclosure is a configuration in which the area estimation unit estimates an attention area in the hidden area adjacent to a living being represented by the image inner area information.

[0163] This makes it easier for the present disclosure to detect an occupant such as an infant, a toddler, and a school-age child.

[0164] An occupant detection method to which this is applied can further exhibit the same effects as described above. Third embodiment

[0165] In the third embodiment, a third example of occupant detection by the occupant detection device 100 is explained.

[0166] In the explanation of the third embodiment, an explanation of the structure explained in the first embodiment or the second embodiment is appropriately omitted.

[0167] The structural example of the occupant detection device 100 according to the third embodiment is the same as the structural example of the occupant detection device 100 explained in the first embodiment, so a detailed explanation of the structural example will be omitted.

[0168] The third example of occupant detection by the occupant detection device 100 will be explained.

[0169] Fig. 14 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the third embodiment, which is a perspective view of a vehicle interior.

[0170] Fig. 15 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the third embodiment, which is a view of a vehicle interior as seen from the front of the vehicle interior.

[0171] Fig. 16 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the third embodiment, which is a view for explaining area estimation processing.

[0172] Fig. 17 is a view for explaining an example of occupant detection by the occupant detection device 100 according to the third embodiment, which is a view for explaining processing in which a living being is detected by the radio intensity by the radio sensor 210.

[0173] An arrow C in Fig. 14 shows a direction seen, for example, from the mounting position of the camera 200.

[0174] The case is being investigated in which there is a living being 620 in the vehicle interior, lying on the rear seats 412.

[0175] In this state, in an image acquired by the image acquisition unit 110, as shown in Fig. As shown in Figure 15, the driver's seat 411-1 of the front seats 411 becomes an obscuring object, creating a blind spot. Likewise, the passenger seat 411-2 of the front seats 411 becomes an obscuring object, creating a blind spot.

[0176] In the image captured by the camera 200, attributes of the living being 620 cannot be detected, although it is included in a partial area 620a that is in contact with the rear seat 412.

[0177] The area identification unit 120 identifies the areas of the driver's seat 411-1 and the passenger seat 411-2 of the front seats 411 and the area of ​​the rear seats 412 as areas of vehicle equipment. Furthermore, the area identification unit 120 identifies the partial area 620a of the living being 620.

[0178] The area estimation unit 130 estimates, as in Fig. 16, an attention area 650.

[0179] Specifically, the area estimation unit 130 estimates whether the partial area 620a in contact with the rear seat 412 is a part of the living being 620 (head, neck, chest, abdomen, hip, thigh, or legs of a living being), and also estimates the attention area 650 of a size corresponding to the entire body structure by estimating the entire body structure based on the part of the living being 620.

[0180] By means of the processing by the wireless sensor data acquisition unit 140 and the detection validity range / threshold optimization unit 150, the living being detection unit 160 acquires Fig. 17 illustrated data.

[0181] The living being detection unit 160 detects the living being 620 with attributes of an infant / toddler or schoolchild by means of the partial area 620a of the living being 620 encompassed in the image and by means of the radio intensities 660 acquired in the attention area 650.

[0182] For example, in Fig. 17, in the occupant detection device 100, the occupant information output unit 170 outputs information that the occupant is exclusively an infant / toddler, information that the occupants are exclusively an infant / toddler or a school-age child, or information about the possibility of being left behind.

[0183] The occupant detection device according to the present disclosure is of a configuration in which the area estimation unit estimates the attention area by estimating an area in which a living being may be located extending across the partial area and the hidden area, using the partial area when the area of ​​living beings represented by the image inner area information is a partial area in contact with the seat surface of a rear seat.

[0184] This makes it easier to detect, for example, a passenger lying on a rear seat.

[0185] An occupant detection method to which this is applied can further exhibit the same effects as described above.

[0186] An example of setting an attention range when a detection target range is preset will now be explained.

[0187] Fig. 18 is a diagram explaining another example of occupant detection by the occupant detection device 100 according to the present disclosure.

[0188] In the occupant detection device 100, before setting an attention range estimated by the range estimation unit 130, detection target ranges 761, 762 and 763 may also be set for each of the Fig. 18. Here, the preset detection target areas 761, 762, and 763 are reset as attention areas by the area estimation unit 130 when an attention area has been estimated by the area estimation unit 130.

[0189] Now, the hardware structure by which the functions of the occupant detection device 100 according to the present disclosure are realized will be explained.

[0190] Fig. 19 is a view showing a first example of a hardware configuration for realizing the functions of the occupant detection apparatus 100 according to the present disclosure.

[0191] Fig. 20 is a view showing a second example of a hardware configuration for realizing the functions of the occupant detection apparatus 100 according to the present disclosure.

[0192] The occupant detection device 100 of the present disclosure is implemented by a Fig. 19 or Fig. 20 hardware shown.

[0193] The occupant detection device 100 comprises, as in Fig. 19, a processor 10001 and a memory 10002.

[0194] The processor 10001 and the memory 10002 are installed, for example, in a computer. The memory 10002 stores programs that provide the computer with functions as an image acquisition unit 110, an area identification unit 120, an area estimation unit 130, a wireless sensor data acquisition unit 140, a detection validity range / threshold optimization unit 150, a living being detection unit 160, an occupant information output unit 170, and a control unit (not shown). The functions of the image acquisition unit 110, the area identification unit 120, the area estimation unit 130, the wireless sensor data acquisition unit 140, the detection validity range / threshold optimization unit 150, the living being detection unit 160, the occupant information output unit 170, and the control unit not shown are realized by reading and executing the programs stored in the memory 10002 by the processor 10001.

[0195] Furthermore, the dictionary data 125 and the unillustrated storage unit are realized by the memory 10002 or another unillustrated memory.

[0196] For example, the processor 10001 uses a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcomputer, or a DSP (Digital Signal Processor).

[0197] The memory 10002 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash memory, a magnetic disk such as a hard disk or a floppy disk, an optical disk such as a CD (Compact Disc) or a DVD or Versatile Disc), or an opto-magnetic disk.

[0198] The processor 10001 and the memory 10002 are connected to each other in a state that enables mutual data transmission. Furthermore, the processor 10001 and the memory 10002 are connected to other hardware via an input / output interface 10003 in a state that enables mutual data transmission.

[0199] The functions of the image acquisition unit 110, the area identification unit 120, the area estimation unit 130, the wireless sensor data acquisition unit 140, the detection validity range / threshold optimization unit 150, the living being detection unit 160, the occupant information output unit 170 and the control unit not shown can also be performed by a special processing circuit 10004 as shown in Fig. 20 shown, can be realized.

[0200] The processing circuit 10004 is, for example, one using a simple circuit, a complex circuit, a programmed processor, parallel programmed processors, ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip) or System LSI (Large Scale Integration).

[0201] Furthermore, the dictionary data 125 and the unillustrated storage unit are realized by a memory 10005 or another unillustrated memory.

[0202] The memory 10005 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash memory, a magnetic disk such as a hard disk or a floppy disk, an optical disk such as a CD (Compact Disc) or a DVD or Versatile Disc), or an opto-magnetic disk.

[0203] The processing circuit 10004 and the memory 10005 are connected to each other in a state that enables mutual data transmission. Furthermore, the processing circuit 1004 and the memory 10005 are connected to other hardware via an input / output interface 10006 in a state that enables mutual data transmission.

[0204] The functions of the image acquisition unit 110, the area identification unit 120, the area estimation unit 130, the wireless sensor data acquisition unit 140, the detection validity range / threshold optimization unit 150, the living being detection unit 160, the occupant information output unit 170, and the control unit not shown can each be realized by individual processing circuits or all by one processing circuit.

[0205] Furthermore, part of the functions of the image acquisition unit 110, the area identification unit 120, the area estimation unit 130, the wireless sensor data acquisition unit 140, the detection validity range / threshold optimization unit 150, the living being detection unit 160, the occupant information output unit 170, and the unillustrated control unit can be realized by the processor 10001 and the memory 10002, and the remaining functions can be realized by the processing circuit 10004.

[0206] In the present disclosure, an unrestricted combination of the respective embodiments and any change of a component of the respective embodiments are possible within the scope of the invention, and further, any component of the respective embodiments may be omitted. Industrial application

[0207] In particular, the occupant detection device according to the present disclosure facilitates the detection of attributes of a living being such as an infant / toddler and school-age child, making it suitable for use as an occupant detection device for a vehicle. Explanation of reference symbols 1 occupant detection system 100 occupant detection device 110 Image Acquisition Section 120 Area Identification Unit 125 dictionary entries 130 area assessment unit 140 Radio Sensor Data Acquisition Unit 150 Proof validity range / threshold optimization unit 160 living organism detection units 170 Occupant information output unit 200 Camera (imaging device) 210 radio sensor 300 occupant information processing device 400 vehicles 410 Vehicle interior 411 front seat 411-1 Driver's seat 411-2 Passenger seat 412 rear seat 415 Hidden area 420 living beings (inmate) 430 Child seat (object on a seat) 440 living beings (inmate) 450 living organism detection area 455 Attention area 460 radio intensity 465 radio intensity 520 living beings (inmate) 540 living beings (inmate) 550 living organism detection area 555 Attention area 560 radio intensity 620 living beings (inmate) 620a sub-area 650 attention area 660 radio intensity 761, 762, 763 Detection target range 10001 processor 10002 memory 10003 Input / output interface 10004 Processing circuit 10005 memory 10006 Input / output interface QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2019-123354 A

[0006]

Claims

[1] Occupant detection device comprising acquires an image acquisition unit for acquiring an image imaged by an imaging device, an area identification unit for outputting image interior area information, by which, in the image acquired by the image acquisition unit, an area of ​​a vehicle interior, an area of ​​a living being and an area of ​​an object on a seat are distinguished, an area estimation unit for estimating, by means of the image interior area information output by the area identification unit, with respect to a hidden area created in the image by the vehicle device, an attention area in which a living being may be located, for each living being concerned, a radio sensor data acquisition unit for acquiring, by means of a result detected by a radio sensor, in time sequence, radio intensity data showing the distribution of the radio intensity of reflection waves, a detection validity range / threshold optimization unit for selecting, from the radio intensity data acquired by the radio sensor data acquisition unit, radio intensity data in the attention range estimated by the range estimation unit, a living being detection unit for detecting, by means of the radio intensity data selected by the detection validity range / threshold optimization unit, an attribute of a living being, and an occupant information output unit for generating, by means of the detection result of the living being detection unit, occupant information representing the occupants of the vehicle, and outputting the occupant information to an occupant information processing device. [2] Occupant detection device according to claim 1, wherein the detection validity range / threshold optimization unit further selects, from the radio intensity data in the attention range estimated by the range estimation unit, radio intensity data exceeding the threshold by means of a threshold corresponding to the range represented by the image inner range information. [3] Occupant detection device according to claim 1 or 2, wherein the area estimation unit estimates an attention area that is smaller than the occluded area. [4] Occupant detection device according to claim 1 or 2, wherein the area of ​​objects on a seat identified by the area identification unit is a sub-area of ​​a child seat, and the area estimation unit estimates the attention area by estimating the entire area of ​​the child seat using the area of ​​objects on a seat identified by the area identification unit. [5] Occupant detection device according to claim 1 or 2, wherein the area estimation unit estimates the attention area in the occluded area adjacent to a living being represented by the image inner area information. [6] Occupant detection device according to claim 1 or 2, wherein the area estimation unit, when the area of ​​living beings represented by the image interior area information is a partial area in contact with the seat surface of a rear seat, uses the partial area to estimate the attention area by estimating an area in which a living being could be located, extending over the partial area and the hidden area. [7] Occupant detection device according to claim 1 or 2, wherein the area estimation unit further determines a living being detection area in which a living being is located by means of image inner area information output by the area identification unit, and the living being detection unit also detects an attribute of the living being using the living being detection area. [8] Occupant detection device according to claim 1, wherein the living being detection unit also detects the number of living beings in the vehicle interior using the living being detection area, attention area and radio intensity. [9] Occupant detection method, comprising an image acquisition step of acquiring, by an image acquisition unit, an image imaged by an imaging device, an area identification step of outputting, by an area identification unit, image inner area information by which in the image, acquired by the image acquisition unit, an area of ​​a vehicle interior, an area of ​​a living being and an area of ​​an object on a seat can be distinguished, an area estimation step of estimating, by an area estimation unit, using the image interior area information output by the area identification unit, an attention area in which a living being may be located with respect to a hidden area created in the image by the vehicle devices, for each living being concerned, a radio sensor data acquisition step of acquiring, by a radio sensor data acquisition unit, radio intensity data showing the distribution of radio intensity of reflection waves by means of a result detected by a radio sensor, in time sequence, a detection validity range / threshold optimization step of selecting, by a detection validity range / threshold optimization unit, from the radio intensity data acquired by the radio sensor data acquisition unit, radio intensity data in the attention range estimated by the range estimation unit, a living being detection step of detecting, by a living being detection unit, using the radio intensity data selected by the detection validity range / threshold optimization unit, an attribute of a living being, and an occupant information output step of generating, by an occupant information output unit, occupant information representing the occupants of the vehicle using the detection result of the living being detection unit, and outputting the occupant information to an occupant information processing device.

Images