Crew identification device and crew identification method

Abstract

To properly determine whether or not an occupant is seated. [Solution] The occupant determination device 11 includes: an acquisition unit 111 that acquires an image PC including an occupant M inside the vehicle 1 and seats S located inside the vehicle 1; a detection unit 112 that, based on the image PC, detects whether each of the seats S located inside the vehicle 1 is flipped up, folded, or tilted at a predetermined angle φA or more as the seat layout LS of the seat S; a setting unit 113 that sets an area R corresponding to a seat S in which an occupant M can sit in the image PC according to the seat layout LS of the seat S; and a determination unit 114 that determines whether an occupant M is located in a seat S based on the area R set by the setting unit 113.

Description

Technical Field

[0004] , ,

[0001] The present invention relates to an occupant determination device and an occupant determination method.

Background Art

[0002] When recognizing an occupant based on an image captured by a camera mounted inside a vehicle, an area where the occupant is predicted to be seated is provided within the image, and it is determined whether the occupant is sitting on the seat or the posture of the occupant, etc., using the area within the image. For example, Patent Document 1 discloses the following content. By collecting a reference image of a passenger sitting on a normal seat, the face area of the passenger at the normal seating position is determined based on the reference image, and based on this, the theoretical seat area is estimated. In this way, the seat area can more realistically reflect the area on the target image that highly correlates with whether the passenger is sitting at a predetermined position.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the seat installed inside the vehicle may bounce up against the wall surface inside the vehicle, or the backrest of the seat may be folded, and the arrangement of the seat may be changed. In order to more appropriately determine the state of the occupant, it is necessary to reflect the change in the seat arrangement in the area as well. In such a case, in the technology described in Patent Document 1 and the like, the area cannot be appropriately set, so there is a possibility that it cannot be appropriately determined whether an occupant is located on the seat. An object of the present invention is to provide an occupant determination device and an occupant determination method capable of appropriately determining whether an occupant is located on a seat. [Means for solving the problem]

[0005] To achieve the above objective, for example, the occupant determination device of this embodiment includes: an acquisition unit that acquires an image including an occupant in a vehicle and seats located in the vehicle; a detection unit that, based on the image, detects whether each of the seats located in the vehicle is flipped up, folded, or tilted beyond a predetermined angle as the seat layout; a setting unit that sets an area in the image corresponding to a seat in which the occupant can sit, according to the seat layout; and a determination unit that determines whether the occupant is located in the seat based on the image corresponding to the area set by the setting unit.

[0006] To achieve the above objective, for example, the occupant determination method of this embodiment includes: an acquisition step of acquiring an image including an occupant in a vehicle and seats located in the vehicle; a detection step of detecting, based on the image, whether each of the seats located in the vehicle is flipped up, folded, or tilted to a predetermined angle or more, as the seat layout; a setting step of setting an area in the image corresponding to a seat in which the occupant can sit, according to the seat layout; and a determination step of determining whether the occupant is located in the seat based on the image corresponding to the area set in the setting step. [Effects of the Invention]

[0007] According to the occupant determination device and occupant determination method of the present invention, it is possible to properly determine whether or not an occupant is located in a seat. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the seating configuration inside a vehicle. [Figure 2] This figure shows an example of the configuration of a crew identification device. [Figure 3] This figure shows an example of the functional configuration of the processor in the crew identification device. [Figure 4] This figure shows an example of the functional configuration of the memory in the crew identification device. [Figure 5] This figure shows an example of the relationship between an image and a region. [Figure 6] This figure shows an example of the relationship between an image containing a seat in a flipped-up position and a region. [Figure 7] This figure shows an example of the relationship between an image containing a folded seat and a region. [Figure 8] This figure shows an example of the relationship between an image containing a reclined seat and a region. [Figure 9] This flowchart shows an example of the processing performed by the crew identification device. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a diagram showing an example of the configuration of a seat S located inside a vehicle 1. The upper part of Figure 1 shows a perspective plan view of the vehicle 1, and the lower part of Figure 1 shows a perspective side view of the vehicle 1. Figure 1 shows the X, Y, and Z axes. The X and Y axes are parallel to the horizontal direction. The Z axis is parallel to the vertical direction. The positive direction of the X axis indicates the right direction of vehicle 1. The positive direction of the Y axis indicates the forward direction of vehicle 1. The positive direction of the Z axis indicates the upward direction of vehicle 1.

[0010] As shown in Figure 1, the vehicle 1 is equipped with seven seats S. The seven seats S are arranged in three rows along the Y-axis. The seven seats S are arranged in the first, second, and third rows, from the positive to the negative direction of the Y-axis. Each of the first, second, and third rows consists of multiple seats S, and these multiple seats S are arranged along the X-axis.

[0011] The first row consists of seat S1 and seat S2. Seat S1 is the so-called "driver's seat," where the driver sits. Seat S2 is the so-called "passenger seat." Seat S2 is positioned in the negative direction of the X-axis relative to seat S1. The second row consists of seat S3, seat S4, and seat S5. Seat S4 is positioned in the negative direction of the X-axis relative to seat S3. Seat S5 is positioned in the negative direction of the X-axis relative to seat S4. The third row consists of seat S6 and seat S7. Seat S7 is positioned in the negative direction of the X-axis relative to seat S6. In the following description, when seats S1-S7 are not distinguished from each other, each of seats S1-S7 may be referred to as seat S.

[0012] Each seat S consists of a seating area SA, a backrest SB, and a headrest SC. The seating area SA is where the occupant sits. The backrest SB supports the occupant M's back. The headrest SC supports the occupant M's head. In this embodiment, occupant M means a person sitting in a seat S in the vehicle 1. Occupant M includes the driver.

[0013] As shown in Figure 1, the occupant determination device 11 is positioned, for example, on the dashboard. The camera 12 is positioned on the upper front of the vehicle 1. The camera 12 generates an image PC that includes the occupant M inside the vehicle 1 and the seats S located inside the vehicle 1, i.e., seats S1-S7. The left-right field of view angle θ1 and the up-down field of view angle θ2 of the camera 12 are set so that the image PC includes the occupant M inside the vehicle 1 and seats S1-S7. The camera 12 includes an image sensor such as a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) and a data processing circuit that generates the image PC from the image sensor.

[0014] FIG. 2 is a diagram showing an example of the configuration of the occupant determination device 11. The occupant determination device 11 is a computer including a processor 11A such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), and a memory 11B such as a ROM (Read Only Memory) or a RAM (Random Access Memory). In addition to these devices, the occupant determination device 11 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), sensors, an interface circuit for connecting peripheral devices, and an in-vehicle network communication circuit for communicating with other in-vehicle devices via an in-vehicle network. The occupant determination device 11 realizes various functional configurations by the processor 11A executing a control program stored in the memory 11B.

[0015] FIG. 3 is a diagram showing an example of the functional configuration of the processor 11A of the occupant determination device 11. FIG. 4 is a diagram showing an example of the functional configuration of the memory 11B of the occupant determination device 11. As shown in FIG. 3, the processor 11A included in the occupant determination device 11 includes, for example, as functional units, an acquisition unit 111, a detection unit 112, a setting unit 113, and a determination unit 114. By the processor 11A executing a control program stored in the memory 11B, the processor 11A functions as the acquisition unit 111, the detection unit 112, the setting unit 113, and the determination unit 114. As shown in FIG. 4, the memory 11B includes an image storage unit 117 and a layout storage unit 118. By the processor 11A executing a control program stored in the memory 11B, the memory 11B functions as the image storage unit 117 and the layout storage unit 118.

[0016] The image storage unit 117 stores an image PC. The image PC is generated by the camera 12. The image PC is acquired from the camera 12 by the acquisition unit 111. The image PC is stored in the image storage unit 117 by the acquisition unit 111.

[0017] The layout storage unit 118 stores in advance various layouts LS of the seat S of the vehicle 1. The various layouts LS include the basic layout LS shown in FIG. 1, the layout LS including the seat S that has been flipped up, the layout LS including the seat S that has been folded, and the layout LS including the seat S that has been tilted by a predetermined angle φA or more. Each of the seat S that has been flipped up, the seat S that has been folded, and the seat S that has been tilted by a predetermined angle φA or more is, for example, any one of the seats S2, S3 - S4, S5, S6, and S7.

[0018] The acquisition unit 111 acquires an image PC obtained by photographing the occupant M in the vehicle 1 and the seat S located in the vehicle 1. The acquisition unit 111 acquires the image PC from the camera 12. The acquisition unit 111 stores the acquired image PC in the image storage unit 117. The image PC includes the occupant M in the vehicle 1 and the seats S1 - S7.

[0019] Based on the image PC, the detection unit 112 detects, for each seat S located in the vehicle 1, whether the seat S has been flipped up, whether the seat S has been folded, and whether the seat S has been tilted by a predetermined angle φA or more, as the layout LS of the seat S. The tilt angle φ of the seat S indicates the angle formed by the seating part SA and the backrest SB. The tilt angle φ is shown in FIG. 1. The predetermined angle φA is, for example, "150 degrees". The detection unit 112 detects whether the tilt angle φ of the seat S has been tilted by a predetermined angle φA or more.

[0020] That the seat S has been flipped up means that the backrest SB of the seat S is stored along the wall in the vehicle 1 in a state where it is folded toward the seating part SA, in other words, it means that it has been flipped. That the seat S has been folded means that the backrest SB of the seat S is folded toward the seating part SA and the seat S is located on the floor in the vehicle 1.

[0021] In the following description, the state in which seat S is flipped up may be described as the flipped-up state ST1. The state in which seat S is folded may be described as the folded state ST2. The state in which seat S is tilted at an angle φA or more may be described as the tilted state ST3. The detection unit 112 detects whether seat S is in the flipped-up state ST1, the folded state ST2, or the tilted state ST3 as the layout LS of seat S. The flipped-up state ST1 will be further explained with reference to Figure 6. The folded state ST2 will be further explained with reference to Figure 7. The tilted state ST3 will be explained further with reference to Figure 8.

[0022] The detection unit 112 may, for example, detect the layout LS of the seat S stored in the layout storage unit 118 based on the image PC. In other words, the detection unit 112 may, for example, detect which of the seat S layout LS stored in the layout storage unit 118 corresponds to the seat S layout LS of the seat S that corresponds to the image PC.

[0023] The setting unit 113 sets the area R corresponding to the seat S in which an occupant M can sit, according to the seat S layout LS, on the image PC. The setting unit 113 does not set the area R corresponding to the flipped-up seat S in the image PC when the seat S is flipped up. The setting unit 113 does not set the area R corresponding to the folded seat S in the image PC when the seat S is folded up. The setting unit 113 does not set the area R corresponding to the seat S that is tilted up to a predetermined angle φA or more in the image PC when the seat S is tilted up to a predetermined angle φA or more. In other words, the setting unit 113 does not set the area R corresponding to the seat S, which is in the flipped-up state ST1, the folded state ST2, or the tilted state ST3, on the image PC.

[0024] When a seat S is flipped up, if a rear seat SR, which is a seat S located behind the flipped-up seat S, is located within the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR in the image PC. When a seat S is folded up, if a rear seat SR, which is a seat located behind the folded seat S, is located within the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR in the image PC. When a seat S is tilted back by a predetermined angle φA or more, if a rear seat SR, which is a seat located behind the seat S tilted back by a predetermined angle φA or more, is located within the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR in the image PC. In other words, when a rear seat SR, which is a seat S located behind a seat S that is in a flipped-up state ST1, a folded state ST2, or a tilted state ST3, is located within the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR in the image PC in the direction in which the seating portion SA of the rear seat SR is located. This direction is sometimes referred to as "forward" in this embodiment. "Forward" refers to the positive direction of the Y-axis as shown in Figure 1, and corresponds to the downward direction in the image PC shown in Figures 5-7.

[0025] The determination unit 114 determines whether or not an occupant M is located in the seat S based on the image PC corresponding to the region R set by the setting unit 113. The determination unit 114 detects characteristic points of the occupant M's body captured in the image PC and compares the coordinates of the characteristic points in the image PC with the region R to determine whether or not an occupant M is sitting in the seat S corresponding to the region R. For example, the determination unit 114 determines that an occupant M is sitting in the seat S corresponding to the region R when the coordinates of the characteristic points in the image PC are included in the region R at a predetermined rate. For example, the determination unit 114 determines that an occupant M is not sitting in the seat S corresponding to the region R when the coordinates of the characteristic points in the image PC are not included in the region R at a predetermined rate.

[0026] One method for determining whether or not an occupant M is located in seat S is the one described in "George Papandreou, et.al., PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model". This method uses a convolutional neural network to detect parts of the human body such as wrists, shoulders, and head as keypoints, and then estimates the positions of other nearby keypoints from each detected keypoint, ultimately determining whether or not an occupant M is located in seat S.

[0027] Figure 5 shows an example of the relationship between image PC1 and region R. Image PC1 is image PC when the seat layout LS is the basic layout. In Figures 5-8, the image of crew member M included in image PC is omitted for convenience.

[0028] The image PC1 shown in the upper part of Figure 5 contains images of seats S1-S7. Each of seats S1-S7 is in a state where an occupant M can sit. The region image RC1 shown in the lower part of Figure 5 shows an example of region R set by the setting unit 113 for each of seats S1 to S7 in image PC1. Regions R11 to R71 each correspond to seats S1 to S7 in image PC1.

[0029] Figure 6 shows an example of the relationship between image PC2, which includes seat S in the flipped-up state ST1, and region R. As shown in the upper image PC2 of Figure 6, seats S3-S5 are in the flipped-up state ST1. In the flipped-up state ST1, the seating area SA is positioned approximately parallel to the YZ plane and along the side of the vehicle body 1.

[0030] The region image RC2 shown in the lower part of Figure 6 shows an example of region R set by the setting unit 113 for each of seats S1 to S7 in image PC2. Since seats S3 to S5 are in the flipped-up state ST1, the setting unit 113 does not set region R for them.

[0031] The seats S6-S6 located behind the seats S3-S5, which are in the flipped-up position ST1, correspond to the rear seat SR. The setting unit 113 extends the region R corresponding to each of the seats S6-S6 forward in the image PC2. As a result, as shown by the shading in the region image RC2, the setting unit 113 extends the regions R61-R71 corresponding to each of the seats S6-S6 forward in the image PC2, such as the regions R62-R72. "Forward" refers to the positive direction of the Y-axis as shown in Figure 1, and corresponds to the downward direction in image PC2 shown in Figure 6. The setting unit 113 expands region R61, for example, based on the image PC, so that region R62 includes the seat portion SA, backrest SB, and headrest SC that constitute the seat S6. Similarly, the setting unit 113 expands region R71, for example, based on the image PC, so that region R72 includes the seat portion SA, backrest SB, and headrest SC that constitute the seat S7.

[0032] Figure 7 shows an example of the relationship between image PC3, which includes the seat S in the folded state ST2, and region R. As shown in image PC2 in the upper part of Figure 7, the seat S2 is in the folded state ST2. In the folded state ST2, the backrest SB is folded approximately parallel to the XY plane.

[0033] The region image RC3 shown in the lower part of Figure 7 shows an example of a region R set by the setting unit 113 for each of the seats S1 to S7 in image PC3. Since seat S2 is in the folded state ST2, the setting unit 113 does not set a region R corresponding to seat S2 in region image RC3.

[0034] The seat S5 located behind the seat S2, which is in the folded state ST2, corresponds to the rear seat SR. Therefore, the setting unit 113 extends the region R corresponding to seat S5 forward in the image PC3. As a result, as shown by the shading in the region image RC3, the setting unit 113 extends the region R51 (see Figure 5) corresponding to seat S5 forward in the image PC3, as shown in region R53.

[0035] In Figure 5, region R51 is set so as not to include the seating area SA of the rear seat SR. This is because the seating area SA of seat 51 is not captured by the image PC due to seat R21, which is located in front of seat 51, which is the rear seat SR in Figure 5. On the other hand, in Figure 7, seat S2 is folded, so the seating area SA of the rear seat SR is captured in image PC3. For this reason, the setting unit 113 expands the area R corresponding to seat S5 to include the seating area SA of the rear seat SR.

[0036] "Forward" refers to the positive direction of the Y-axis as shown in Figure 1, and corresponds to the downward direction in image PC3 shown in Figure 7. The setting unit 113 expands region R51, for example, based on the image PC, so that region R53 includes the seat portion SA, backrest SB, and headrest SC that constitute the seat S6.

[0037] Figure 8 shows an example of the relationship between image PC4, which includes seat S in tilted state ST3, and region R. As shown in image PC3 in the upper part of Figure 6, seat S2 is in tilted state ST3. In tilted state ST3, the tilt angle φ of seat S is tilted to, for example, "180 degrees".

[0038] The region image RC4 shown in the lower part of Figure 8 shows an example of a region R set by the setting unit 113 for each of the seats S1 to S7 in image PC4. Since seat S2 is in the tilted state ST3, the setting unit 113 does not set a region R corresponding to seat S2 in region image RC4.

[0039] The seat S5 located behind the seat S2, which is in the tilted position ST3, corresponds to the rear seat SR. The setting unit 113 extends the region R corresponding to seat S5 forward in the image PC4. As a result, as shown by the shading in the region image RC4, the setting unit 113 extends the region R51 (see Figure 5) corresponding to seat S5 forward in the image PC4, as shown by the region R54. Similar to the case where the rear seat SR is in the folded position ST2, the setting unit 113 extends the region R corresponding to seat S5 to a size that includes the seating area SA of the rear seat SR. "Forward" refers to the positive direction of the Y-axis as shown in Figure 1, and corresponds to the downward direction in image PC4 shown in Figure 8. The setting unit 113 expands region R51 in image PC4, for example, so that region R53 includes the seat SA, backrest SB, and headrest SC that constitute the seat S5.

[0040] Figure 9 is a flowchart showing an example of the processing of the occupant determination device 11. First, in step S101, the acquisition unit 111 acquires an image PC from the camera 12. In step S103, the detection unit 112 selects one seat S from among a plurality of seats S located in the vehicle 1, which in this embodiment are seven seats S, based on the image PC. In step S105, the detection unit 112 detects whether the seat S selected in step S105 is in the flipped-up state ST1 based on the image PC. If the detection unit 112 detects that the seat S is in the flipped-up state ST1 (step S105; YES), the process proceeds to step S113. If the detection unit 112 detects that the seat S is not in the flipped-up state ST1 (step S105; NO), the process proceeds to step S107.

[0041] In step S107, the detection unit 112 detects, based on the image PC, whether the seat S selected in step S103 is in the folded state ST2. If the detection unit 112 detects that the seat S is in the folded state ST2 (step S107; YES), the process proceeds to step S113. If the detection unit 112 detects that the seat S is not in the folded state ST2 (step S107; NO), the process proceeds to step S109.

[0042] In step S109, the detection unit 112 detects, based on the image PC, whether the seat S selected in step S103 is in the tilted state ST3. If the detection unit 112 detects that the seat S is in the tilted state ST3 (step S109; YES), the process proceeds to step S113. If the detection unit 112 detects that the seat S is not in the tilted state ST3 (step S109; NO), the process proceeds to step S111. In step S111, the setting unit 113 sets the area R corresponding to the seat S selected in step S103 to the image PC. Then the process proceeds to step S115.

[0043] If the answer is YES in step S105, YES in step S107, and YES in step S109, in step S113, the setting unit 113 does not set the area R corresponding to the seat S selected in step S103 to the image PC. In step S115, the detection unit 112 determines whether all seven seats S located in the vehicle 1 have been selected. If the detection unit 112 determines that not all seven seats S have been selected (step S115; NO), the process returns to step S103. If the detection unit 112 determines that all seven seats S have been selected (step S115; YES), the process proceeds to step S117.

[0044] In step S117, the setting unit 113 determines whether the rear seat SR, which is a seat S located behind a seat S that is not set in the image PC, is located within the vehicle 1. If the setting unit 113 determines that the rear seat SR is not located within the vehicle 1 (step S117; NO), the process proceeds to step S121. If the setting unit 113 determines that the rear seat SR is located within the vehicle 1 (step S117; YES), in step S119, the setting unit 113 extends the area R corresponding to the rear seat SR forward in the image PC. In step S121, the determination unit 114 detects characteristic points of the occupant M's body captured in the image PC. In step S123, the determination unit 114 determines whether the occupant M is located in the seat S based on the image PC corresponding to the area R set by the setting unit 113. After that, the process ends.

[0045] Step S101 corresponds to an example of an "acquisition step". Steps S105, S107, and S109 correspond to an example of a "detection step". Steps S111 and S113 correspond to an example of a "setting step". Step S121 corresponds to an example of a "decision step".

[0046] As described above with reference to Figures 1 to 9, the occupant determination device 11 according to this embodiment includes: an acquisition unit 111 that acquires an image PC including an occupant M in the vehicle 1 and seats S located in the vehicle 1; a detection unit 112 that, based on the image PC, detects whether each of the seats S located in the vehicle 1 is flipped up, folded, or tilted at a predetermined angle φA or more as the seat layout LS of the seat S; a setting unit 113 that sets a region R corresponding to a seat S in which an occupant M can sit in the image PC according to the seat layout LS of the seat S; and a determination unit 114 that determines whether an occupant M is located in a seat S based on the region R set by the setting unit 113.

[0047] The occupant determination method according to this embodiment includes: an acquisition step of acquiring an image PC including an occupant M inside the vehicle 1 and seats S located inside the vehicle 1; a detection step of detecting, based on the image PC, whether each of the seats S located inside the vehicle 1 is flipped up, folded, or tilted at a predetermined angle φA or more, as the seat layout LS of the seat S; a setting step of setting a region R corresponding to a seat S in which an occupant M can sit in the image PC according to the seat layout LS of the seat S; and a determination step of determining whether an occupant M is located in a seat S based on the region R set in the setting step.

[0048] As the seat layout LS, for each seat S located within the vehicle 1, it is detected whether the seat S is flipped up, whether the seat S is folded up, and whether the seat S is tilted at a predetermined angle φA or more. In accordance with the seat layout LS, the area R corresponding to a seat S in which an occupant M can sit is set on the image PC, so the area R can be set appropriately. Based on the area R, it is determined whether or not an occupant M is located in the seat S, so it can be determined appropriately whether or not an occupant M is located in the seat S.

[0049] In the occupant determination device 11, the setting unit 113 does not set the area R corresponding to the flipped-up seat S in the image PC when the seat S is flipped up, does not set the area R corresponding to the folded seat S in the image PC when the seat S is folded up, and does not set the area R corresponding to the seat S that is tilted up to a predetermined angle φA or more in the image PC when the seat S is tilted up to a predetermined angle φA or more.

[0050] In the above-described occupant determination method, when a seat S is flipped up, the region R corresponding to the flipped-up seat S is not set in the image PC; when a seat S is folded, the region R corresponding to the folded seat S is not set in the image PC; and when a seat S is tilted back by a predetermined angle φA or more, the region R corresponding to the seat S tilted back by a predetermined angle φA or more is not set in the image PC.

[0051] Since the image PC does not set regions R corresponding to seats S that are flipped up, folded up, or tilted at an angle φA or more, region R can be set appropriately. Therefore, it is possible to properly determine whether or not an occupant M is located in a seat S.

[0052] In the occupant determination device 11, the setting unit 113 expands the area R corresponding to the rear seat SR forward in the image PC when a seat S is flipped up and a rear seat SR located behind the flipped-up seat S is located inside the vehicle 1. When a seat S is folded up and a rear seat SR located behind the folded seat S is located inside the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR forward in the image PC. When a seat S is tilted back by a predetermined angle φA or more and a rear seat SR located behind the tilted seat S is located inside the vehicle 1, the setting unit 113 expands the area R corresponding to the rear seat SR forward in the image PC.

[0053] In the above occupant determination method, in the setting step, if seat S is flipped up and a rear seat SR located behind the flipped up seat S is located inside the vehicle 1, the area R corresponding to the rear seat SR is expanded forward in the image PC; if seat S is folded up and a rear seat SR located behind the folded seat S is located inside the vehicle 1, the area R corresponding to the rear seat SR is expanded forward in the image PC; and if seat S is tilted down by a predetermined angle φA or more and a rear seat SR located behind the tilted seat S is located inside the vehicle 1, the area R corresponding to the rear seat SR is expanded forward in the image PC.

[0054] The area R corresponding to the rear seat SR, which is located behind the flipped-up seat S, the folded-up seat S, and the seat S that is tilted back by a predetermined angle φA or more, is extended forward. Therefore, the area R can be set appropriately. As a result, it can be properly determined whether or not an occupant M is located in the seat S.

[0055] The embodiment described above is merely an example of one embodiment of the present invention, and can be modified and applied as desired without departing from the spirit of the present invention.

[0056] In this embodiment, the case in which seven seats S are arranged in the vehicle 1 has been described, but the embodiment is not limited to this. For example, the vehicle 1 may have two or more rows of seats S in the front-to-back direction and three or more seats S. For example, the vehicle 1 may have two rows of seats S in the front-to-back direction and four seats S. For example, the vehicle 1 may have three rows of seats S in the front-to-back direction and six seats S.

[0057] In this embodiment, we have described a case in which the vehicle 1 includes seats S3-S5 that can be flipped up ST1, and seat S2 that can be folded up ST2 and tilted ST3, but the embodiment is not limited to this. Vehicle 1 only needs to have at least one seat S that can be in one of three states: a flipped-up state ST1, a folded-up state ST2, or a tilted state ST3. For example, Vehicle 1 may have only a seat S that can be flipped up ST1. For example, Vehicle 1 may have only a seat S that can be folded up ST2. For example, Vehicle 1 may have only a seat S that can be tilted ST3.

[0058] Vehicle 1 may be configured to include a seat S that can be in one state and a seat S that can be in any one other state. The seat S that can be in one state is at least one seat S that can be in one of the following states: flipped up state ST1, folded state ST2, and tilted state ST3. The seat S that can be in any one other state is at least one seat S that can be in any one other state than flipped up state ST1, folded state ST2, and tilted state ST3. Vehicle 1 may include, for example, a seat S that can be in flipped up state ST1 and a seat S that can be in folded state ST2. Vehicle 1 may include, for example, a seat S that can be in flipped up state ST1 and a seat S that can be in tilted state ST3. Vehicle 1 may include, for example, a seat S that can be in folded state ST2 and a seat S that can be in tilted state ST3.

[0059] In this embodiment, a case in which the layout LS of the seat S is changed by placing the seat S in a flipped-up state ST1, a folded state ST2, or a tilted state ST3 has been described, but the embodiment is not limited to this. For example, at least one seat S may be detachably configured to be attached to the main body of the vehicle 1. For example, all seats S2-S7 other than the driver's seat S1 may be detachably configured. In this case, the variations in the seat layout LS can be diversified. For example, at least one seat S may be configured to be retractable into and out of the main body of the vehicle 1. For example, all seats S2-S7 other than the driver's seat S1 may be configured to be retractable. In this case, the variations in the seat layout LS can be diversified.

[0060] In this embodiment, the setting unit 113 extends forward the region R corresponding to the rear seat SR, which is a seat S located behind the seat S in the flipped-up state ST1, folded state ST2, or tilted state ST3; however, the embodiment is not limited to this. The setting unit 113 only needs to extend forward the region R corresponding to the rear seat SR, which is a seat S located behind the seat S in the tilted state ST3.

[0061] For example, Figures 2-4 are diagrams that classify the components according to their main processing content in order to facilitate understanding of the present invention. The components can be further classified into more components according to their processing content. A single component can also be classified to perform even more processing. The processing of each component may be executed on one piece of hardware or on multiple pieces of hardware. The processing of each component may be implemented by one program or by multiple programs.

[0062] In this embodiment, the crew determination device 11 comprises an acquisition unit 111, a detection unit 112, a setting unit 113, a determination unit 114, and an image storage unit 117, but is not limited thereto. A server device that is communicably connected to the crew determination device 11 via a network such as the Internet may comprise, for example, at least one of the detection unit 112, the setting unit 113, and the determination unit 114. The server device may comprise, for example, the detection unit 112. In this case, the load on the crew determination device 11 can be reduced.

[0063] When the crew determination device 11 of the present invention is implemented using a computer, the control program executed by the computer may be configured as a recording medium or a transmission medium for transmitting the control program. Magnetic, optical, or semiconductor memory devices can be used as the recording medium. Examples include portable or fixed recording media such as flexible disks, HDDs, CD-ROMs (Compact Disk Read Only Memory), DVDs, Blu-ray® Discs, magneto-optical disks, flash memory, and card-type recording media. The above-mentioned recording media may also be non-volatile storage devices such as RAM, ROM, or HDDs provided by the crew determination device 11. The control program may be downloaded by the crew determination device 11 from a server device that is communicated with the crew determination device 11 via a network.

[0064] For example, the processing units in the flowchart shown in Figure 9 are divided according to their main processing content to facilitate understanding of the processing of the crew determination device 11, and the present invention is not limited by the way the processing units are divided or their names. The processing of the crew determination device 11 may be further divided into more processing units depending on the processing content. The processing of the crew determination device 11 may also be divided so that one processing unit includes even more processing. [Explanation of Symbols]

[0065] 1...Vehicle, 11...Occupant determination device, 11A...Processor, 111...Acquisition unit, 112...Detection unit, 113...Setting unit, 114...Determination unit, 11B...Memory, 116...Control program, 117...Image storage unit, 118...Layout storage unit, 12...Camera, LS...Layout, M...Occupant, PC, PC1-PC4...Image, RC1-RC4...Region image, R...Region, S, S1-S7...Seat, SA...Seating area, SB...Backrest, SC...Headrest, SR...Rear seat, ST1...Folded state, ST2...Folded state, ST3...Tilted state, θ1, θ2...Field of view angle, φ...Incline angle, φA...Determined angle.

Claims

1. An acquisition unit that acquires images of the occupants inside the vehicle and the seats located inside the vehicle, Based on the aforementioned image, a detection unit detects, as the seat layout, whether each seat located inside the vehicle is flipped up, folded down, or tilted beyond a predetermined angle. A setting unit sets the area in the image to correspond to the seats in which the occupants can sit, according to the seat layout. A determination unit that determines whether or not the occupant is positioned in the seat based on the region set by the setting unit, A crew identification device equipped with the following features.

2. The aforementioned setting unit is, When the aforementioned seat is flipped up, the area corresponding to the flipped-up seat is not set in the image. When the aforementioned seat is folded, the area corresponding to the folded seat is not set in the image. When the seat is reclined beyond a predetermined angle, the area corresponding to the seat reclined beyond the predetermined angle is not included in the image. Crew determination device according to claim 1.

3. The aforementioned setting unit is, When the aforementioned seat is folded up, if a rear seat located behind the folded-up seat is located within the vehicle, the area corresponding to the rear seat is expanded in the image. When the aforementioned seat is folded, if a rear seat, which is located behind the folded seat, is located within the vehicle, then in the image, the area corresponding to the rear seat is expanded. When the aforementioned seat is reclined beyond a predetermined angle, and a rear seat located behind the seat reclined beyond the predetermined angle is located within the vehicle, the area corresponding to the rear seat is expanded in the image. Crew determination device according to claim 2.

4. Acquisition step: Acquisition step of acquiring an image including the occupants inside the vehicle and the seats located inside the vehicle, A detection step is performed to detect, based on the aforementioned image, whether each seat located in the vehicle is flipped up, whether it is folded up, and whether it is reclined beyond a predetermined angle, as the seat layout. Setting step: In accordance with the seat layout, set the area in the image to correspond to the seats in which the occupants can sit. A determination step in which, based on the area set in the setting step, determines whether or not the occupant is positioned in the seat, Crew identification method, including

5. In the above setting step, When the aforementioned seat is flipped up, the area corresponding to the flipped-up seat is not set in the image. When the aforementioned seat is folded, the area corresponding to the folded seat is not set in the image. When the seat is reclined beyond a predetermined angle, the area corresponding to the seat reclined beyond the predetermined angle is not included in the image. The crew determination method according to claim 4.

6. In the above setting step, When the aforementioned seat is folded up, if a rear seat located behind the folded-up seat is located within the vehicle, the area corresponding to the rear seat is expanded in the image. When the aforementioned seat is folded, if a rear seat, which is located behind the folded seat, is located within the vehicle, then in the image, the area corresponding to the rear seat is expanded. When the aforementioned seat is reclined beyond a predetermined angle, and a rear seat located behind the seat reclined beyond the predetermined angle is located within the vehicle, the area corresponding to the rear seat is expanded in the image. The crew determination method according to claim 5.

Images