Boarding position determination device, system, method, and program

Abstract

The present invention makes it possible to identify a seat for an occupant to ride in. A camera (130) is installed in a vehicle in which a plurality of rows of seats are disposed, and images the inside of the vehicle. A face detecting unit (111) detects the facial region of an occupant from the images captured by the camera (130). A riding row identifying unit (112) identifies the seat row in which the occupant, whose facial region has been detected, is riding. A riding position identifying unit (113) identifies the seat position of the occupant in the vehicle on the basis of a range of seat positions prepared for each seat row in the image and the identified seat row.

Description

[Technical Field]

[0001] The present disclosure relates to a ride position determination device, system, method, and computer-readable medium. [Background technology]

[0002] As a related technique, Patent Document 1 discloses an airbag device and a method for controlling its deployment. In Patent Document 1, a camera captures an image of an occupant sitting in a seat where an airbag is installed. An occupant recognition unit performs image processing on the image captured by the camera to determine the position of the occupant present in the image. A distance recognition unit determines the distance from a reference position to the occupant. A position detection unit detects the actual position of the occupant sitting in the seat based on the position of the occupant on the image and the distance from the reference position determined by the distance recognition unit to the occupant. A deployment control unit determines whether the occupant's head is located within the airbag deployment range. If the occupant's head is located within the airbag deployment range, the deployment control unit deploys the airbag. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-001125 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the position detection unit detects the actual position of the occupant sitting in the seat. However, in Patent Document 1, the position of the occupant's head is detected for comparison with the airbag deployment range, and the seat in which the occupant is sitting is not identified.

[0005] In view of the above circumstances, one of the objects of the present disclosure is to provide a seating position determination device, system, method, and computer-readable medium that can identify which seat an occupant is sitting in. [Means for solving the problem]

[0006] To achieve the above object, the present disclosure provides, as a first aspect, a riding position determination device, which includes: a face detection unit that detects a facial area of ​​an occupant from an image captured by a camera installed inside a vehicle having multiple rows of seats; a riding row identification unit that identifies the seat row in which the occupant whose facial area has been detected is riding; and a riding position identification unit that identifies the seat position of the occupant in the vehicle based on a range of seat positions in the image, which is provided for each seat row, and the identified seat row.

[0007] In a second aspect, the present disclosure provides a passenger position determination system. The passenger position determination system includes a camera installed in a vehicle with multiple rows of seats and capturing images of the interior of the vehicle, and a seat position determination device that acquires images captured by the camera and identifies the seat positions of passengers in the vehicle using the acquired images. The seat position determination device includes face detection means that detects a facial area of ​​the passenger from the image, a passenger row identification means that identifies the seat row in which the passenger whose facial area is detected is seated, and a passenger position identification means that identifies the seat position of the passenger in the vehicle based on a range of seat positions in the image, which is provided for each seat row, and the identified seat row.

[0008] In a third aspect, the present disclosure provides a riding position determination method, which includes detecting a facial area of ​​an occupant from an image captured by a camera installed inside a vehicle having multiple rows of seats, identifying a seat row in which the occupant whose facial area is detected is riding, and identifying the seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the identified seat row.

[0009] In a fourth aspect, the present disclosure provides a computer-readable medium storing a program for causing a processor to execute a process including detecting a facial region of an occupant from an image captured by a camera installed inside a vehicle having multiple rows of seats, identifying a seat row in which the occupant whose facial region is detected is seated, and identifying a seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the identified seat row. [Effects of the Invention]

[0010] The passenger position determination device, system, method, and computer-readable medium according to the present disclosure can identify which seat an occupant is sitting in from an image captured by a camera. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a riding position determination system according to a first embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram showing a vehicle on which a camera is installed. [Figure 3] FIG. 10 is a schematic diagram showing the relationship between a detected face area and the range of each seat. [Figure 4] 4 is a flowchart showing an operation procedure of the boarding position determination device. [Figure 5] FIG. 10 is a block diagram showing a riding position determination device according to a second embodiment of the present disclosure. [Figure 6] FIG. 1 is a block diagram showing a content distribution system in which a riding position determination device is used. [Figure 7] 10 is a flowchart showing an operation procedure of a riding position determination device according to a third embodiment of the present disclosure. [Figure 8] 10 is a flowchart showing an operation procedure of a riding position determination device according to a fourth embodiment of the present disclosure. [Figure 9] FIG. 2 is a block diagram showing the hardware configuration of a riding position determination device 110. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the following description and drawings have been omitted and simplified as appropriate for clarity of explanation. In addition, in each drawing, the same or similar elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary.

[0013] FIG. 1 shows a riding position determination system according to a first embodiment of the present disclosure. The riding position determination system 100 includes a riding position determination device 110 and a camera 130. The camera 130 captures images of the interior of a vehicle having multiple rows of seats. The camera 130 is installed, for example, between the driver's seat and the passenger seat in a position that allows a view of the entire interior of the vehicle. The riding position determination device 110 acquires the image captured by the camera 130 and, based on the acquired image, identifies in which seat the occupant of the vehicle is sitting.

[0014] FIG. 2 shows a vehicle in which a camera 130 is installed. The vehicle 200 is a moving object such as a passenger car, a taxi, or a van. In the vehicle 200, the camera 130 is installed, for example, at a position such as the base of the rearview mirror, facing the interior of the vehicle 200. The capturing range of the camera 130 includes the area of ​​the seats provided in the vehicle. For example, if the vehicle has two rows of seats, and the front row (row 1) can accommodate two people and the rear row (row 2) can accommodate three people, the camera 130 captures an area including a total of five seats (seats 0-4). Note that the number of cameras 130 in the riding position determination system 100 is not limited to one. The riding position determination system 100 may include multiple cameras 130 arranged in one vehicle.

[0015] Information about the seat arrangement in the vehicle cabin is registered in the riding position determination device 110. For example, information indicating how many rows of seats are provided in the vehicle 200 and information about the number of seats in each row are registered in the riding position determination device 110. Information indicating the distance from the camera 130 to each row of seats and the width of each seat is also registered in the riding position determination device 110. The riding position determination device 110 detects occupants from the video captured by the camera 130 and identifies in which seat the detected occupant is sitting. The riding position determination device 110 is mounted on, for example, the vehicle 200. Alternatively, the riding position determination device 110 may be a device installed outside the vehicle. In this case, the riding position determination device 110 may acquire the video captured by the camera 130 via a wireless communication network.

[0016] The riding position determination device 110 has a face detection unit 111, a riding train identification unit 112, and a riding position identification unit 113. The riding position determination device 110 is configured as a device including, for example, one or more processors and one or more memories. At least some of the functions of each unit in the riding position determination device 110 can be realized by the processor performing processing in accordance with a program read from the memory.

[0017] The face detection unit (face detection means) 111 detects a facial area of ​​a subject (occupant) from a video (image) captured by the camera 130. When an image includes multiple subjects, the face detection unit 111 detects multiple facial areas. For example, when the face detection unit 111 detects a new facial area, it assigns a tracking ID (Identifier) ​​to the detected facial area. The face detection unit 111 tracks the position of the detected facial area in the video in the time direction for each tracking ID. The method of detecting and tracking the facial area in the face detection unit 111 is not limited to a specific method. For example, the face detection unit 111 can detect a facial area using face authentication technology. The face detection unit 111 may also track the same facial area using position information of the facial area.

[0018] The train identification unit (train identification means) 112 identifies the seat row in which the occupant whose face area has been detected is riding. For example, the train identification unit 112 estimates the distance from the position in the vehicle 200 where the face area exists to the camera 130 for each tracking ID. In other words, the train identification unit 112 estimates the distance from the position of the occupant's face to the camera 130 for each occupant. For example, the train identification unit 112 estimates the distance from the position of the occupant's face to the camera 130 based on the distance between feature points in the face area. For each occupant, the train identification unit 112 estimates the seat row (train) in which the occupant is riding based on the estimated distance.

[0019] For example, the train identification unit 112 extracts the eyes of the passenger as feature points and obtains the distance between the eyes on the image. For example, the train identification unit 112 assumes that the distance between the eyes is a predetermined value, for example, 6 cm (=0.06 m), and estimates the distance (position) of the passenger in the real space in the depth direction based on the distance between the eyes on the image, the angle of view θ of the camera 130, and the number of pixels D in the horizontal direction. Specifically, if the distance E between the eyes is 6 cm (=0.06 m), and the distance between the eyes on the image is D, e The distance H from the eyes to the back of the head of a person is set to a predetermined value of 30 cm (=0.3 m). When a pinhole camera model is used, the distance Y in the real space in the depth direction of the occupant is e In the case of the central projection method, it can be calculated by the following formula. TIFF0007732519000001.tif2059 In the case of equidistant projection, the distance Y in the real space in the depth direction of the occupant e can be calculated by the following formula: TIFF0007732519000002.tif2049 Above distance Y e The equation is a simple equation that does not take into account lens distortion, etc. The train identification unit 112 calculates the distance Y e Just calculate it.

[0020] The train identification unit 112 calculates the distance Y eand the distance from each seat row to the camera 130. For example, the seat row identification unit 112 may e and the distance from each seat row to the camera 130 is calculated. The seat row identification unit 112 identifies the seat row in which the occupant is riding based on the calculated difference. For example, the seat row identification unit 112 identifies the seat row in which the occupant is riding with the smallest difference as the seat row in which the occupant is riding. If the seats are configured to be adjustable in terms of reclining angle and fore-aft position, the seat row identification unit 112 may acquire seat control information from the vehicle and change the distance from each seat row to the camera 130 in accordance with the acquired control information. For example, the seat row identification unit 112 may acquire seat reclining angles from the vehicle and calculate the distance from each seat row to the camera 130 based on the acquired reclining angles.

[0021] The method used by the train identification unit 112 to estimate the depth of the passenger is not limited to the above-described method. For example, the train identification unit 112 may estimate the depth of the passenger using feature points other than the eyes. Alternatively, the train identification unit 112 may estimate the depth of the passenger from the image captured by the camera 130 using depth estimation based on deep learning. If the camera 130 is a stereo camera, the train identification unit 112 may estimate the depth of the passenger using a parallax image. Furthermore, the train identification unit 112 may acquire distance information from a time-of-flight (ToF) camera or a ranging device.

[0022] The seat position specifying unit (seat position specifying means) 113 stores information indicating the position (area) of each seat on the camera image for each seat row. The position of each seat on the camera image (area of ​​the seat position) is (a) Calculate the coordinates of the seat position in the bird's-eye view in the camera coordinate system with the camera as the origin. (b) The camera's angle of view θ and the number of horizontal pixels D, and the distance Y between the camera and each seat row in real space s Based on the relationship between the vehicle width and the seat position on the image, the coordinate D in the vehicle width direction (X direction) s Calculate When the pinhole camera model is used, the coordinates D of each seat position in the vehicle width direction are calculated as s In the case of the central projection method, it can be calculated using the following formula. TIFF0007732519000003.tif2040Also, in the case of the equidistant projection method, the coordinate D of each seat position in the vehicle width direction (X direction) s can be calculated using the following formula: TIFF0007732519000004.tif2043In the above formula, θ s represents the angle between the line connecting the position (origin) of the camera 130 and the headrest of each seat, and the longitudinal direction of the vehicle (Y axis). s is a simple formula that does not take into account lens distortion, etc. The riding position identification unit 113 calculates the coordinates D s The method for identifying the position of each seat in the camera image is not limited to the above-mentioned method.

[0023] For example, in the case where there are two seat rows, the seat ranges on the image for each seat in the front row and each seat in the back row are registered in the riding position identification unit 113. For example, when it is determined that an occupant is riding in the back row, the riding position identification unit 113 compares the ranges of seats 0, 1, and 2 (see FIG. 2) with the range of the face area. When it is determined that an occupant is riding in the front row, the riding position identification unit 113 compares the ranges of seats 3 and 4 with the range of the face area. For each seat, the riding position identification unit 113 calculates the rate at which the seat position range and the face area overlap (overlap rate), and identifies which seat the occupant is sitting in based on the magnitude of the overlap rate.

[0024] FIG. 3 schematically shows the relationship between the detected face area and the range of each seat. In this example, it is assumed that the occupant is identified as riding in the front row. The riding position identification unit 113 calculates the overlap rate between the face area and the seat range, that is, the degree to which the range of the face area overlaps with the range of each seat. In the example of FIG. 3, the overlap rate between the face area and the range of seat 3 is 100%, and the overlap rate between the face area and the range of seat 4 is 0%. In this case, the riding position identification unit 113 identifies the occupant as riding in seat 3.

[0025] If it is determined that the passenger is riding in the back row, the riding position identification unit 113 calculates the overlap rate between the face area and the range of seat 0, the overlap rate between the face area and the range of seat 1, and the overlap rate between the face area and the range of seat 2. The riding position identification unit 113 identifies that the passenger is riding in the seat with the highest overlap rate among seat 0, seat 1, and seat 2.

[0026] Next, the operation procedure will be explained. FIG. 4 shows the operation procedure (boarding position determination method) in the boarding position determination device 110. The face detection unit 111 acquires an image from the camera 130 (step A1). The face detection unit 111 detects a face area from the acquired image (step A2). The boarding train identification unit 112 identifies a boarding train for the detected face area (step A3). In step A3, the boarding train identification unit 112 extracts, for example, multiple feature points in the face area. The boarding train identification unit 112 assumes that the distance between the extracted multiple feature points is a constant value, and estimates the position of the face area in the depth direction (the length direction of the vehicle). The boarding train identification unit 112 identifies the boarding train of the face area based on the estimated position in the depth direction.

[0027] The boarding position identification unit 113 identifies the seat position of the occupant based on the range of each seat on the image in the boarding train identified in step A3 and the range of the face area (step A4). In step A4, the boarding position identification unit 113 calculates, for example, the overlap rate between the range of each seat on the image in the identified boarding train and the range of the face area. The boarding position identification unit 113 identifies the seat with the highest overlap rate as the boarding position of the occupant. The boarding position identification unit 113 can output the identified seat position of the occupant to an external device (not shown).

[0028] In this embodiment, the boarding row identification unit 112 identifies the seat row in which the occupant in the face area detected by the face detection unit 111 is riding. The boarding position identification unit 113 identifies the seat position of the occupant in the vehicle based on the range of seat positions on the image prepared for each seat row and the seat row identified by the boarding row identification unit 112. In this manner, the boarding position determination device 110 can identify not only the position of the occupant in the vehicle but also the seat in which the occupant is riding, based on the camera image. In this embodiment, the boarding position determination device 110 can identify the boarding position of the occupant using, for example, an image from one camera 130 installed between the driver's seat and the passenger seat. In this case, only one camera 130 is required, and the boarding position of the occupant can be identified at low cost.

[0029] Next, a second embodiment of the present disclosure will be described. Fig. 5 shows a riding position determination device according to the second embodiment of the present disclosure. The riding position determination device 110 according to this embodiment has a face authentication unit 114 and an attribute acquisition unit 115 in addition to the components of the riding position determination device 110 according to the first embodiment shown in Fig. 1. The face authentication unit (face authentication means) 114 performs face authentication on the detected face area. If the occupant in the detected face area is a person who has been registered in advance, the face authentication unit 114 outputs information that identifies the person as the authentication result.

[0030] The attribute acquisition unit (attribute acquisition means) 115 acquires attribute information of the person identified by the face authentication unit 114. The attribute acquisition unit 115, for example, refers to a database that stores attribute information about a plurality of people, and acquires the attribute information of the person identified by the face authentication unit 114. The attribute information includes, for example, information such as age, gender, occupation, and hobbies. In this embodiment, the riding position identification unit 113 can output information that identifies the occupant or the occupant's attribute information to an external device (not shown), in addition to the seat position of the identified occupant.

[0031] In this embodiment, the face authentication unit 114 performs face authentication on the detected face area to identify the individual. Furthermore, the attribute acquisition unit 115 acquires attribute information of the occupant. In this embodiment, the riding position determination device 110a can not only identify the seat position of the occupant, but also identify who is sitting in which seat. Alternatively, the riding position determination device 110a can identify which person with which attribute information is sitting in which seat.

[0032] In the above description, an example has been described in which the attribute acquisition unit 115 acquires an authentication result from the face authentication unit 114 and acquires attribute information of the person identified by the face authentication unit 114. However, the present embodiment is not limited to this. The attribute acquisition unit 115 may acquire attribute information such as age group and gender from the face area detected by the face detection unit 111, for example.

[0033] The riding position determination device 110a according to this embodiment can be used, for example, in a content distribution system. FIG. 6 shows a content distribution system in which the riding position determination device 110a is used. The content distribution system 300 includes the riding position determination device 110a, a content distribution device 310, and multiple monitors 320-340. In the content distribution system 300, the monitor 320 is assumed to be, for example, a monitor for seat 4 (passenger seat) shown in FIG. 2. The monitor 330 is assumed to be, for example, a monitor for seat 0 shown in FIG. 2. The monitor 340 is assumed to be, for example, a monitor for seat 2 shown in FIG. 2.

[0034] The content distribution device 310 acquires information indicating the seat in which the occupant is sitting from the riding position determination device 110a. Alternatively, the content distribution device 310 acquires information indicating whether or not an occupant is sitting in each seat from the riding position determination device 110a. Furthermore, the content distribution device 310 acquires information identifying the person sitting in each seat or attribute information of the person sitting in each seat from the riding position determination device 110a. The content distribution device 310 outputs content to the monitors 320-340. The content output to the monitors 320-340 includes, for example, advertising content and video content.

[0035] For example, the content distribution device 310 outputs content to a monitor corresponding to a seat occupied by a passenger. The content distribution device 310 does not need to output content to a monitor corresponding to an empty seat. When information identifying a person occupying a seat is acquired, the content distribution device 310 may output content customized for the identified person to a monitor. When attribute information of the person occupying a seat is acquired, the content distribution device 310 may output content according to the acquired attribute information to a monitor. The content distribution device 310 may distribute general content to a monitor corresponding to a seat for which a person is not identified or for which attribute information is not acquired.

[0036] The information identifying the person riding in each seat or the attribute information of the person riding in each seat acquired by the riding position determination device 110a may also be used for control of the vehicle 200. The control of the vehicle 200 may include, for example, adjusting the reclining angle or fore-aft position of the seat, and setting the temperature or air volume of the air conditioning device. For example, the vehicle 200 may adjust the reclining angle or fore-aft position of the seat according to the information identifying the person riding in the front row seat or the attribute information of the person riding in the front row seat. Alternatively, the vehicle 200 may change the settings of the air conditioning device according to the information identifying the person riding in each seat or the attribute information of the person riding in each seat.

[0037] Next, a third embodiment of the present disclosure will be described. The configuration of the riding position determination device according to this embodiment is similar to the configuration of the riding position determination device 110 according to the first embodiment shown in Fig. 1. The configuration of the riding position determination device according to this embodiment may be similar to the configuration of the riding position determination device 110a according to the second embodiment shown in Fig. 5.

[0038] In the first embodiment, when the train identification unit 112 estimates the distance in the depth direction by utilizing the fact that the distance between feature points is a constant value, the distance between feature points may become shorter when a passenger looks to the side, and the estimated distance in the depth direction may become longer than the actual distance. For example, when a passenger sitting in the front row seat closest to the camera looks to the side, the distance between their eyes in the image may become half or less of the actual distance. In this case, the train identification unit 112 may erroneously identify the train of the passenger sitting in the front row as the rear row because the estimated distance in the depth direction becomes longer than the actual distance. This embodiment is an embodiment that at least partially solves such a problem.

[0039] In this embodiment, the face detection unit 111 assigns a tracking ID to a face area when it detects it, and tracks the face area for each tracking ID. If a passenger in the front row tries to move to the back row, it is considered that the passenger will turn around while moving. In this case, it is considered that the detection of the face area will be temporarily interrupted, and the face area will not be able to move from the front row to the back row while maintaining the same tracking ID. In this embodiment, the passenger train identification unit 112 counts the number of times a passenger's train is identified as the front row for each passenger. If a passenger's train is identified as the front row a certain number of times (front row count) or more, the passenger train identification unit 112 identifies the passenger as riding in the front row as long as the passenger's face area continues to be detected continuously.

[0040] It is assumed that for passengers in the rear row, the distance between feature points will not change by more than two times depending on the direction of their faces. Therefore, it is assumed that passengers in the rear row will not be mistakenly recognized as passengers in the front row. Furthermore, in the above example, when the distance in the depth direction is estimated based on the distance between feature points in the face region, a passenger in the front row is mistakenly recognized as passengers in the rear row. However, in this embodiment, the estimation of the distance in the depth direction is not limited to estimation based on the distance between feature points in the face region. This embodiment can be useful for cases other than estimating the distance in the depth direction based on the distance between feature points in the face region.

[0041] 7 shows the operation procedure of the riding position determination device 110 in this embodiment. The face detection unit 111 acquires an image from the camera 130 (step B1). The face detection unit 111 detects a face area from the acquired image (step B2). Steps B1 and B2 may be similar to steps A1 and A2 shown in FIG. 4.

[0042] The train identification unit 112 determines whether the face area detected in step B2 is a new face area (step B3). In other words, the train identification unit 112 determines whether the face area detected in step B2 is a face area that has been continuously detected since before, or a newly detected face area. The train identification unit 112 determines whether the detected face area is a new face area based on, for example, a tracking ID assigned to the face area by the face detection unit 111.

[0043] If the boarding train identification unit 112 determines in step B3 that the detected face area is a new face area, it identifies a boarding train for the detected face area (step B4). Step B4 may be the same as step A3 shown in FIG. 4. If the boarding train identification unit 112 determines in step B3 that the detected face area is not a new face area, that is, if it determines that the detected face area is a face area that has already been detected, it determines whether the front row count is equal to or greater than a predetermined value (step B5). If the boarding train identification unit 112 determines in step B5 that the front row count is equal to or greater than the predetermined value, it identifies that the passenger is riding in the front row (step B6). If the boarding train identification unit 112 determines in step B5 that the front row count is not equal to or greater than the predetermined value, it proceeds to step B4 and identifies a boarding train for the detected face area.

[0044] The train identification unit 112 determines whether the passenger is identified as riding in the front row (step B7). If the train identification unit 112 determines in step B7 that the passenger is identified as riding in the front row, it adds 1 to the front row count (step B8). If the train identification unit 112 determines in step B7 that the passenger is identified as not riding in the front row, it resets the front row count (step B9).

[0045] The boarding position identification unit 113 identifies the boarding positions of the passengers based on the range of each seat on the image in the boarding train identified in step B4 or B6 and the range of the face area (step B10). Step B10 may be the same as step A4 shown in FIG.

[0046] In this embodiment, the train identification unit 112 counts the number of times that a passenger in a continuously detected face area is identified as riding in the front row as a front row count. If the front row count is equal to or greater than a predetermined value, the train identification unit 112 identifies the passenger's train row as the front row. In this case, a passenger who has been identified as riding in the front row a predetermined number of times or more is identified as riding in the front row as long as face detection continues to be successful. Therefore, this embodiment can prevent a passenger riding in the front row from being mistakenly identified as riding in the back row. Other effects are the same as those of the first or second embodiment.

[0047] A fourth embodiment of the present disclosure will now be described. The configuration of the riding position determination device according to this embodiment is similar to the configuration of the riding position determination device 110 according to the first embodiment shown in Fig. 1. The configuration of the riding position determination device according to this embodiment may be similar to the configuration of the riding position determination device 110a according to the second embodiment shown in Fig. 5.

[0048] In the third embodiment, it was assumed that a passenger in a rear row would not be mistakenly identified as being in the front row. However, if a passenger in a rear row moves sideways while remaining in the rear row, their posture may lean forward, shortening the estimated depth distance. For example, when the passenger train identification unit 112 estimates the depth distance using the fact that the distance between feature points is a constant value, the distance between the feature points of the passenger in the rear row may be shortened. Specifically, when a passenger leans forward, the distance between their eyes in the image may be half or more of the actual distance. In this case, the estimated depth distance becomes shorter than the actual distance, and the passenger train identification unit 112 may mistakenly identify the passenger train of the passenger in the rear row as the front row. In this case, the assumption assumed in the third embodiment that a passenger in a rear row would not be mistakenly identified as being in the front row does not hold. This embodiment is an embodiment that at least partially solves the above problem.

[0049] In this embodiment, even if the boarding train is identified as the front row, the train identification unit 112 does not add 1 to the front row count if the occupant is moving. For example, the train identification unit 112 determines whether the occupant is moving based on whether the position of the face area has changed. More specifically, the train identification unit 112 determines whether the occupant is moving based on, for example, the distance between the center of the face detection frame, which is a frame indicating the face area, and the center of the face detection frame a predetermined number of frames before. For example, the train identification unit 112 may determine whether the occupant is moving based on whether the distance between the centers of the face detection frames is equal to or less than a predetermined percentage of the width of the face detection frame (equal to or less than L%, where L is a positive integer). For example, the train identification unit 112 determines that the occupant is moving if the distance between the centers of the face detection frames exceeds L% of the width of the face detection frame. If the occupant is moving, the train identification unit 112 resets the front row count without adding any additional count. This prevents the front row count from exceeding a predetermined value while the passenger is moving, preventing a passenger in a rear row from being mistakenly identified as a passenger in a front row.

[0050] 8 shows the operation procedure of the boarding position determination device 110 in this embodiment. The face detection unit 111 acquires an image from the camera 130 (step C1). The face detection unit 111 detects a face area from the acquired image (step C2). The boarding train identification unit 112 determines whether the face area detected in step C2 is a new face area (step C3). The boarding train identification unit 112 determines whether the detected face area is a new face area based on, for example, a tracking ID assigned to the face area by the face detection unit 111.

[0051] If the boarding train identification unit 112 determines in step C3 that the detected face area is a new face area, it identifies the boarding train for the detected face area (step C4). If the boarding train identification unit 112 determines in step C3 that the detected face area is not a new face area, it determines whether the front row count is equal to or greater than a predetermined value (step C5). If the boarding train identification unit 112 determines in step C5 that the front row count is equal to or greater than the predetermined value, it identifies that the passenger is riding in the front row (step C6). If the boarding train identification unit 112 determines in step C5 that the front row count is not equal to or greater than the predetermined value, it proceeds to step C4 and identifies the boarding train for the detected face area.

[0052] The train identification unit 112 determines whether the passenger is identified as riding in the front row (step C7). Steps C1-C7 may be the same as steps B1-B7 shown in FIG. 7. If the train identification unit 112 determines in step C7 that the passenger is identified as not riding in the front row, it resets the front row count (step C8). If the train identification unit 112 determines in step C7 that the passenger is identified as riding in the front row, it determines whether the passenger is moving (step C9). If the train identification unit 112 determines in step C9 that the passenger is not moving, it adds 1 to the front row count (step C10). If the train identification unit 112 determines in step C9 that the passenger is moving, it proceeds to step C8 and resets the front row count.

[0053] The boarding position identification unit 113 identifies the boarding position of the passenger based on the range of each seat on the image in the boarding train identified in step C4 or C6 and the range of the face area (step C11). Steps C8, C10, and C11 may be similar to steps B9, B8, and B10 shown in FIG. 7, respectively.

[0054] In this embodiment, the train identification unit 112 determines whether the position of the face area of ​​a passenger (face area) identified as being in the front row has changed. If the train identification unit 112 determines that the position of the face area has changed, it does not increment the front row count. By doing so, even if a passenger riding in a rear seat is mistakenly identified as being in the front row while moving, the front row count can be prevented from exceeding a predetermined value and continuing to be identified as being in the front row. Other effects are similar to those of the third embodiment.

[0055] Next, the hardware configuration of the riding position determination device 110 will be described. Fig. 9 shows the hardware configuration of the riding position determination device 110. The riding position determination device 110 has a processor (CPU: Central Processing Unit) 501, a ROM (read only memory) 502, and a RAM (random access memory) 503. In the riding position determination device 110, the processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. Although not shown, the riding position determination device 110 may include other circuits such as peripheral circuits, communication circuits, and interface circuits.

[0056] The ROM 502 is a non-volatile storage device. For example, a semiconductor storage device with a relatively small capacity, such as a flash memory, is used for the ROM 502. The ROM 502 stores the programs executed by the processor 501.

[0057] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, ROM, flash memory, solid-state drive (SSD) or other memory technologies, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0058] The RAM 503 is a volatile storage device. Various semiconductor memory devices such as a dynamic random access memory (DRAM) or a static random access memory (SRAM) are used for the RAM 503. The RAM 503 can be used as an internal buffer for temporarily storing data and the like.

[0059] The processor 501 loads the program stored in the ROM 502 into the RAM 503 and executes the program. When the CPU 501 executes the program, the functions of the respective units in the riding position determination device 110 can be realized.

[0060] The above describes the embodiments of the present disclosure in detail, but the present disclosure is not limited to the above-described embodiments, and changes and modifications to the above-described embodiments that do not deviate from the spirit of the present disclosure are also included in the present disclosure.

[0061] For example, some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes.

[0062] [Appendix 1] a face detection means for detecting a facial region of an occupant from an image captured by a camera installed inside a vehicle having a plurality of rows of seats; a seat row identification means for identifying a seat row in which the occupant whose face area has been detected is seated; A passenger position determination device comprising a passenger position identification means for identifying the seat position of the passenger in the vehicle based on the range of seat positions in the image prepared for each seat row and the identified seat row.

[0063] [Appendix 2] The passenger position determination device described in Appendix 1, wherein the passenger position identification means identifies the seat position of the occupant based on the position of the face area and the position range of each of a plurality of seats in the identified seat row.

[0064] [Appendix 3] The passenger position determination device described in Appendix 1 or 2, wherein the passenger position identification means identifies the passenger's seat position based on the overlap ratio between the face area and the position range of each of multiple seats in the identified seat row.

[0065] [Appendix 4] The riding position determination device according to claim 3, wherein the riding position identification means identifies the seat with the largest ratio in the identified seat row as the seat position of the occupant.

[0066] [Appendix 5] the seat rows include a first row that is closer to the camera in the longitudinal direction of the vehicle, and a second row that is located behind the first row; the face detection means tracks the detected face area in a time direction; The passenger row identification means counts the number of times the passenger is identified as riding in the first row as a front row count, and if the front row count of a passenger whose face area is continuously detected by the face detection means is equal to or greater than a predetermined value, the passenger position determination device described in any one of Appendices 1 to 4 is identified as riding in the first row.

[0067] [Appendix 6] The passenger position determination device according to claim 5, wherein the passenger row identification means resets the front row count when it identifies that the passenger is riding in the second row.

[0068] [Appendix 7] The passenger row identification means, when it is determined that the passenger is riding in the first row, determines whether the position of the face area has changed, and when it is determined that the position of the face area has changed, resets the front row count.

[0069] [Appendix 8] The passenger train identification means extracts a plurality of feature points from the image of the face area, estimates a longitudinal position of the vehicle based on the distance between the extracted plurality of feature points, and identifies the seat based on the estimated longitudinal position.

[0070] [Appendix 9] The passenger train identification means assumes that the distances between the plurality of feature points in real space are constant, and estimates the longitudinal position of the vehicle based on the distances between the extracted plurality of feature points and the distances between the plurality of feature points in real space.

[0071] [Appendix 10] A riding position determination device as described in any one of appendix 1 to 9, further comprising a facial recognition means for performing facial recognition on an image of the detected facial area and identifying the occupant whose facial area is detected.

[0072] [Appendix 11] 11. The riding position determination device according to any one of claims 1 to 10, further comprising an attribute acquisition means for acquiring attribute information of the occupant whose face area is detected.

[0073] [Appendix 12] The passenger row identification means acquires control information of the seat and identifies the seat row in which the passenger is riding using the acquired control information of the seat.

[0074] [Appendix 13] a camera installed in a vehicle having a plurality of rows of seats and configured to capture an image of the interior of the vehicle; a seat position determination device that acquires an image captured by the camera and identifies a seat position of an occupant in the vehicle using the acquired image, The seat position determination device a face detection means for detecting a face area of ​​the occupant from the image; a seat row identification means for identifying a seat row in which the occupant whose face area has been detected is seated; A passenger position determination system having a passenger position identification means for identifying the seat position of the occupant in the vehicle based on the range of seat positions in the image prepared for each seat row and the identified seat row.

[0075] [Appendix 14] The passenger position determination system described in Appendix 13, wherein the passenger position identification means identifies the seat position of the occupant based on the position of the face area and the position range of each of a plurality of seats in the identified seat row.

[0076] [Appendix 15] The passenger position determination system described in Appendix 13 or 14, wherein the passenger position identification means identifies the passenger's seat position based on the overlap ratio between the face area and the position range of each of multiple seats in the identified seat row.

[0077] [Appendix 16] A facial region of an occupant is detected from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; A passenger position determination method that includes identifying the seat position of the occupant in the vehicle based on the range of seat positions in the image prepared for each seat row and the identified seat row.

[0078] [Appendix 17] A facial region of an occupant is detected from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; A non-transitory computer-readable medium storing a program for causing a processor to execute a process including identifying the seat position of the occupant in the vehicle based on the range of seat positions in the image prepared for each seat row and the identified seat row. [Explanation of symbols]

[0079] 100: Boarding position determination system 110: Boarding position determination device 111: Face detection unit 112: Boarding line identification section 113: Boarding location identification unit 114: Face recognition unit 115: Attribute acquisition part 130: Camera 200: Vehicle 300: Content distribution system 310: Content distribution device 320-340: Monitor 501: Processor 502:ROM 503:RAM

Claims

1. a face detection means for detecting a face area of ​​an occupant from an image captured by a camera installed inside a vehicle having a plurality of rows of seats; a seat row identification means for identifying a seat row in which the passenger whose face area is detected is seated; a seat position specifying means for specifying a seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the specified seat row; The passenger position identification means is a passenger position determination device that identifies the passenger's seat position based on the overlap ratio between the face area and the position ranges of each of a plurality of seats in the identified seat row.

2. The riding position determination device according to claim 1 , wherein the riding position identification means identifies the seat position of the occupant based on the position of the face area and the position ranges of each of a plurality of seats in the identified seat row.

3. The riding position determination device according to claim 1 or 2, wherein the riding position specifying means specifies the seat with the largest ratio in the specified seat row as the seat position of the occupant.

4. A face detection means for detecting a facial area of ​​an occupant from an image captured by a camera installed inside a vehicle having multiple rows of seats; a seat row identification means for identifying a seat row in which the passenger whose face area is detected is seated; a seat position specifying means for specifying a seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the specified seat row; the seat rows include a first row that is closer to the camera in the longitudinal direction of the vehicle, and a second row that is located behind the first row; the face detection means tracks the detected face area in a time direction; The passenger row identification means counts the number of times the passenger is identified as riding in the first row as a front row count, and if the front row count of a passenger whose face area is continuously detected by the face detection means is equal to or greater than a predetermined value, the passenger position determination device identifies the passenger as riding in the first row.

5. The riding position determination device according to claim 4 , wherein the riding row identification means resets the front row count when it is determined that the occupant is riding in the second row.

6. The passenger row identification means, when it identifies that the occupant is riding in the first row, determines whether the position of the face area has changed, and if it determines that the position of the face area has changed, resets the front row count.

7. 7. The boarding position determination device according to claim 1, wherein the boarding row identification means extracts a plurality of feature points from the image of the face area, estimates a longitudinal position of the vehicle based on the distance between the extracted plurality of feature points, and identifies the seat based on the estimated longitudinal position.

8. 8. The passenger position determination device according to claim 7, wherein the passenger train identification means assumes that the distances between the plurality of feature points in real space are constant, and estimates the longitudinal position of the vehicle based on the distances between the extracted plurality of feature points and the distances between the plurality of feature points in real space.

9. The riding position determination device according to claim 1 , further comprising a face authentication unit that performs face authentication on the image of the detected face area and identifies the occupant whose face area is detected.

10. The riding position determination device according to claim 1 , further comprising an attribute acquisition unit that acquires attribute information of the occupant whose face area is detected.

11. The passenger position determination device according to claim 1 , wherein the passenger row identification means acquires the seat control information and identifies the seat row in which the passenger is riding using the acquired seat control information.

12. a camera installed in a vehicle having a plurality of rows of seats and configured to capture an image of the interior of the vehicle; a seat position determination device that acquires an image captured by the camera and identifies a seat position of an occupant in the vehicle using the acquired image, The seat position determination device a face detection means for detecting a face area of ​​the occupant from the image; a seat row identification means for identifying a seat row in which the passenger whose face area is detected is seated; a seat position specifying means for specifying the seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the specified seat row; The passenger position identification means identifies the passenger's seat position based on the overlap ratio between the face area and the position ranges of each of a plurality of seats in the identified seat row.

13. A camera installed in a vehicle having multiple rows of seats and capturing images of the interior of the vehicle; a seat position determination device that acquires an image captured by the camera and identifies a seat position of an occupant in the vehicle using the acquired image, The seat position determination device a face detection means for detecting a face area of ​​the occupant from the image; a seat row identification means for identifying a seat row in which the passenger whose face area is detected is seated; a seat position specifying means for specifying the seat position of the occupant in the vehicle based on a range of seat positions in the image, which is prepared for each seat row, and the specified seat row; the seat rows include a first row that is closer to the camera in the longitudinal direction of the vehicle, and a second row that is located behind the first row; the face detection means tracks the detected face area in a time direction; The passenger row identification means counts the number of times the passenger is identified as riding in the first row as a front row count, and if the front row count of a passenger whose face area is continuously detected by the face detection means is equal to or greater than a predetermined value, the passenger is identified as riding in the first row.

14. A computer comprising: A facial region of an occupant is detected from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; identifying a seat position of the occupant in the vehicle based on a range of seat positions in the image prepared for each seat row and the identified seat row; A seat position determination method for identifying the seat position of the occupant based on the overlap ratio between the face area and the position ranges of each of a plurality of seats in the identified seat row.

15. A computer comprising: A facial region of an occupant is detected from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; identifying a seat position of the occupant in the vehicle based on a range of seat positions in the image prepared for each seat row and the identified seat row; the seat rows include a first row that is closer to the camera in the longitudinal direction of the vehicle, and a second row that is located behind the first row; tracking the detected face region in a time direction; The number of times the occupant is identified as riding in the first row is counted as a front row count, and if the front row count of an occupant whose face area is continuously detected is equal to or greater than a predetermined value, the occupant is identified as riding in the first row.

16. A facial region of an occupant is detected from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; identifying a seat position of the occupant in the vehicle based on a range of seat positions in the image prepared for each seat row and the identified seat row; A program for causing a processor to execute a process for identifying the seat position of the occupant based on the overlap ratio between the face area and the position ranges of each of a plurality of seats in the identified seat row.

17. A method for detecting facial areas of occupants from an image captured by a camera installed inside a vehicle having multiple rows of seats, Identifying a seat row in which the passenger whose face area is detected is seated; identifying a seat position of the occupant in the vehicle based on a range of seat positions in the image prepared for each seat row and the identified seat row; the seat rows include a first row that is closer to the camera in the longitudinal direction of the vehicle, and a second row that is located behind the first row; tracking the detected face region in a time direction; A program for causing a processor to execute a process of counting the number of times the occupant is identified as sitting in the first row as a front row count, and identifying the occupant as sitting in the first row if the front row count of the occupant whose face area is continuously detected is equal to or greater than a predetermined value.

Images