Vehicle seat system

Abstract

To provide a vehicle seat system that is able to control an operation of a movable device, based on biological information of an occupant (an expression of the occupant).SOLUTION: A vehicle seat system S includes: a vehicle seat 1 having a seat body on which an occupant can be seated, and a movable device (a height device 50) that moves a part or the whole of the seat body; and a control device 100 that operates the movable device, based on a facial expression of the occupant seated on the vehicle seat. The control device 100 includes: a facial expression recognition unit that recognizes the facial expression of the occupant; a facial expression determination unit that determines a type of the facial expression of the occupant, based on recognition information obtained by the facial expression recognition unit; and an operation control unit that controls an operation of the movable device, based on determination information obtained by the facial expression determination unit.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a vehicle seat system, and more particularly to a vehicle seat system including a vehicle seat and a control device that controls the vehicle seat. [Background technology]

[0002] Conventionally, a vehicle seat system has been known which includes a vehicle seat in which an occupant can be seated and a physical information detection unit which detects physical information of the occupant seated in the vehicle seat, and which controls the operation of the vehicle seat's reclining mechanism (seat back tilt mechanism) and a slide rail device (seat translation mechanism) based on the occupant's physical information (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] China Patent Application Publication No. 114475371 Summary of the Invention [Problem to be solved by the invention]

[0004] Meanwhile, in vehicle seat systems such as that described in Patent Document 1, there has been a demand for technology that can control the operation of seat movable devices such as height devices, reclining devices, and slide rail devices based on biometric information (changes in biometric information) that is different from the occupant's physical information. For example, there has been a demand for technology that allows occupants to stay comfortable in a vehicle seat by operating a seat moving device based on changes in the occupant's biometric information.

[0005] An object of the present invention is to provide a vehicle seat system capable of controlling the operation of a seat moving device based on biological information of an occupant (for example, facial expression of the occupant). Another object of the present invention is to provide a vehicle seat system that allows an occupant seated in a vehicle seat to feel comfortable inside the vehicle. [Means for solving the problem]

[0006] The above problem is solved by the vehicle seat system of the present invention, which comprises a vehicle seat having a seat body on which an occupant can sit and a movable device that moves part or all of the seat body, and a control device that controls the operation of the movable device, wherein the control device has an expression recognition unit that recognizes the expression of an occupant seated in the vehicle seat, an expression determination unit that determines the type of expression of the occupant based on recognition information by the expression recognition unit, and an operation control unit that controls the operation of the movable device based on determination information by the expression determination unit. With the above configuration, it is possible to realize a vehicle seat system capable of controlling the operation of the seat moving device based on changes in the biological information of the occupant. More specifically, the control device of this system recognizes the facial expression of the occupant, determines the type of facial expression of the occupant based on the recognition information, and controls the operation of the movable device based on the determination information. In other words, the operation of the seat movable device can be controlled according to the type of facial expression (change in facial expression) of the occupant. Therefore, the seat configuration of the vehicle seat can be arranged so that the occupant can spend more comfortably inside the vehicle. It can also improve entertainment value.

[0007] In this case, the facial expression recognition unit recognizes the facial expressions of multiple occupants, the facial expression determination unit determines the degree of change in facial expression for each occupant based on the recognition information for each occupant, and the operation control unit controls the movable device to move the seat body in which a specific occupant is seated based on the type of facial expression for each occupant and the degree of change in the facial expression determined by the facial expression determination unit. With the above configuration, the seat body in which a specific passenger sits can be operated based on the type of facial expression and the degree of change in the facial expression. For example, it is possible to appropriately operate the seat body in which the passenger who shows the largest smile sits, thereby improving entertainment value.

[0008] In this case, the vehicle seat is provided with an imaging device that images an occupant seated in the vehicle seat, the control device has a physique information detection unit that detects physique information of the occupant, and the operation control unit controls the operation of the movable device based on the physique information of the occupant obtained by the physique information detection unit, and moves the seat body so that the seating position corresponds to the imaging range of the imaging device. With the above configuration, the seat body can be moved so that the seating position corresponds to the imaging range of the imaging device based on the physique information of the occupant, making it easier to recognize the facial expression (smile) of the occupant.

[0009] In this case, the movable device is a height device that connects the seat body to the vehicle floor so that it can be raised and lowered, the control device has a calculation unit that calculates the number of times a predetermined facial expression is determined by the facial expression determination unit, and the operation control unit controls the height device to raise the seat body based on the number of times the predetermined facial expression is determined calculated by the calculation unit. With the above-described configuration, the height device can be operated in accordance with the frequency and size of a predetermined facial expression (for example, a smile) of the occupant, and the seat body can be raised by the height device.

[0010] In this case, the control device is provided with a display device used by an occupant seated in the vehicle seat, and has an output unit that outputs first display information in which the facial expression of the occupant determined by the facial expression determination unit is converted into a character's facial expression, and second display information based on operation information of the movable device controlled by the operation control unit, and the display device displays the first display information and the second display information output by the output unit toward the occupant. With the above configuration, a predetermined facial expression (for example, a smile) of the passenger can be displayed on the display unit using the character, and the movable amount of the movable device can be displayed.

[0011] In this case, the control device is provided with a display device used by an occupant seated in the vehicle seat, and the control device has a calculation unit that calculates the number of times a predetermined facial expression has been determined by the facial expression determination unit, and an output unit that outputs display information based on the number of times the predetermined facial expression has been determined by the calculation unit, and the display device displays the display information output by the output unit toward the occupant. With the above configuration, the number of times the passenger makes a predetermined facial expression (e.g., a smile) can be displayed on the display unit. This can improve entertainment value. For example, a smile can be quantified, which can improve entertainment value.

[0012] In this case, the control device has a calculation unit that calculates the number of times a predetermined facial expression is determined by the facial expression determination unit, and a movement amount determination unit that determines the amount of movement of the seat body based on the number of times the predetermined facial expression is determined by the calculation unit, and the operation control unit controls the movable device based on the amount of movement of the seat body determined by the movement amount determination unit, and moves the seat body. With the above configuration, the movable device can be moved based on the number of times the occupant makes a predetermined facial expression (e.g., a smile), and other occupants can know that the occupant is smiling from the amount of movement of the movable device.

[0013] In this case, a voice acquisition device is provided for acquiring the voice of an occupant seated in the vehicle seat, and the operation control unit controls the movable device to move the seat body based on the smiling facial expression of the occupant determined by the facial expression determination unit and the degree of the occupant's voice obtained by the voice acquisition device. With the above configuration, it is possible to more accurately determine whether the occupant is laughing, and the movable device can be operated based on the determination result.

[0014] In this case, a voice acquisition device is provided for acquiring the voice of an occupant seated in the vehicle seat, and a vibration device is provided for imparting vibrations to the occupant seated in the vehicle seat, and the operation control unit controls the operation of the vibration device based on the level of the occupant's voice acquired by the voice acquisition device. With the above-mentioned configuration, it is possible to recognize that a passenger is laughing by the vibration action of the vibration device, thereby enhancing the entertainment value inside the vehicle.

[0015] In this case, a collision detection unit is provided which detects a collision of the vehicle, and when the collision detection unit detects a collision of the vehicle, the operation control unit controls the movable device not to operate regardless of the judgment information by the facial expression judgment unit. With the above configuration, the operation of the movable device can be stopped if a vehicle collision is predicted.

[0016] In this case, the movable device is a height device that connects the seat body to the vehicle floor so that it can be raised and lowered, and the control device has a physique information detection unit that detects physique information of an occupant seated in the vehicle seat, and the operation control unit controls the height device to limit the raising and lowering movement of the seat body based on the physique information of the occupant. With the above-described configuration, the movement of the height device can be restricted based on the physique information of the occupant, allowing the occupant to be seated comfortably. Effect of the Invention

[0017] According to the present invention, it is possible to realize a vehicle seat system capable of controlling the operation of a seat movable device based on a change in the biological information of an occupant. Also, it is possible to arrange the seat configuration so that the occupant can feel comfortable in the vehicle. Also, it is possible to improve the entertainment value. According to the present invention, the seat body in which a specific occupant is seated can be operated based on the type of facial expression and the degree of change in the facial expression. For example, it is possible to appropriately operate the seat body in which the occupant who shows the largest smile is seated. According to the present invention, the seat body can be moved so that the seating position corresponds to the imaging range of the imaging device based on the physique information of the occupant. For example, it becomes easier to recognize the smiling face of the occupant. Furthermore, according to the present invention, the seat body can be raised in accordance with the number of times the occupant makes a predetermined facial expression (for example, a smile). Furthermore, according to the present invention, a predetermined facial expression (for example, a smile) of the passenger can be displayed on the display unit using the character, and the movable amount of the movable device can be displayed. Furthermore, according to the present invention, the number of times the passenger shows a predetermined facial expression can be displayed. For example, smiling can be quantified, which can improve entertainment value. Furthermore, according to the present invention, it is possible to know whether the occupant is smiling from the amount of movement of the movable device. Furthermore, according to the present invention, it is possible to more accurately determine whether an occupant is laughing, and to operate the movable device based on the determination result. Furthermore, according to the present invention, it is possible to recognize whether an occupant is laughing by the vibration action of the vibration device. The present invention also provides for the ability to disable movement of movable equipment if a vehicle collision is predicted. Furthermore, according to the present invention, the movement of the height device can be restricted based on the physique information of the occupant. [Brief description of the drawings]

[0018] [Figure 1] 1 is an overall view of a vehicle seat system. [Diagram 2] FIG. 2 is a rear view of the vehicle seat showing the position of the imaging device. [Diagram 3] FIG. 11 is a rear view of the vehicle seat of the modified example, showing the position of the imaging device. [Figure 4] 1 is a block diagram showing a hardware configuration and functions of a vehicle seat system. [Diagram 5] FIG. 2 is a side view of a vehicle seat, a display device, and an imaging device. [Figure 6] FIG. 4 is a diagram showing an example of a display screen of a display device. [Figure 7] FIG. 2 is a side view of a vehicle seat, a display device, and an imaging device. [Figure 8] FIG. 4 is a diagram showing an example of a display screen of a display device. [Figure 9] FIG. 2 is a side view of a vehicle seat, a display device, and an imaging device. [Figure 10] 4 is a flowchart illustrating an example of a process of the vehicle seat system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. This embodiment relates to a "vehicle seat system" that is capable of controlling the operation of a seat moving device based on the biological information of an occupant (facial expression of the occupant). The "front-rear direction of the vehicle" refers to the front-rear direction of the vehicle when it is normally running. The "inner side of the width of the vehicle" refers to the interior side of the vehicle (inside the vehicle) relative to the vehicle door, and the "outer side of the width of the vehicle" refers to the exterior side of the vehicle (outside the vehicle) relative to the vehicle door.

[0020] As shown in Fig. 1, the vehicle seat system S is a device that is mounted on a vehicle V and moves a part or the whole of a vehicle seat 1 (seat body 1a) in which a passenger sits, based on the passenger's facial expression. With this system, the seat configuration of the vehicle seat 1 can be adjusted so that the passenger can feel more comfortable inside the vehicle V, for example, as shown in Figs. 5 to 8. In addition, the entertainment value inside the vehicle V can be increased. Specifically, the vehicle seat system S recognizes the facial expression of an occupant seated in the vehicle seat 1. Then, based on the recognition information, it determines the type of facial expression of the occupant (e.g., a smile, etc.). Then, based on the determination information, it controls the operation of the movable devices 40, 50, and 60. For example, it raises the seat body 1a by the height device 50. This allows the seat configuration of the vehicle seat 1 in which the passenger sits to be adjusted, and also allows the entertainment value within the vehicle V to be improved.

[0021] <Hardware configuration of vehicle seat system> The vehicle seat system S includes a plurality of vehicle seats 1 each arranged inside a vehicle V, a plurality of imaging devices 70, a plurality of display devices 80, a plurality of audio acquisition devices 85, a plurality of vibration devices 90, a collision detection sensor 95, and a control device 100. Vehicle V is a vehicle (automobile) as shown in FIG. The vehicle seat 1 includes a front seat 1A (driver's seat), a front seat 1B (passenger's seat), a rear seat 1C, and a rear seat 1D. Each vehicle seat 1 comprises a seat body 1a having a seat cushion 10, a seat back 20 and a headrest 30, a slide rail device 40 that supports the seat body 1a so that it can move back and forth relative to the vehicle floor V1, a height device 50 that connects the seat body 1a to the vehicle floor V1 so that it can be raised and lowered, and a reclining device 60 that connects the seat back 20 to the seat cushion 10 so that it can be rotated. The seat cushion 10, the seat back 20 and the headrest 30 correspond to a seat body 1a. The slide rail device 40, the height device 50 and the reclining device 60 correspond to a movable device.

[0022] As shown in Figures 1 and 2, the seat cushion 10 is a seating portion that supports a seated person (occupant) from below, and includes a cushion frame 11 that forms the skeleton, a padding material 12 that is placed on the cushion frame 11, and a cover material 13 that covers the cushion frame 11 and the padding material 12. The cushion frame 11 is made of a rectangular frame-like body that serves as a skeleton of the seat cushion 10. The cushion frame 11 comprises left and right side frames 11a arranged on the sides in the seat width direction, a plate-shaped pan frame 11b connecting the front portions of each side frame 11a, a pipe-shaped rear connecting frame 11c connecting the rear portions of each side frame 11a, and a number of elastic springs 11d that are hooked onto the pan frame 11b and the rear connecting frame 11c, respectively, and extend in a snake-like shape.

[0023] As shown in FIGS. 1 and 2, the seat back 20 is a backrest portion that supports a seated occupant from behind, and includes a back frame 21 that serves as a skeleton, a padding material 22, and a cover material 23. The back frame 21 is a rectangular frame-like body that serves as the framework of the seat back 20 . The back frame 21 comprises side frames 21a arranged laterally in the seat width direction, an inverted U-shaped upper frame 21b connecting the upper end portions of each side frame 21a, a plate-shaped lower frame 21c connecting the lower end portions of each side frame 21a, a plate-shaped connecting frame 21d connecting the upper portions of each side frame 21a, a plurality of wire members 21e hooked to the connecting frame 21d and the lower frame 21c, and a support plate 21f held by the plurality of wire members 21e and supporting a seated occupant. Left and right pillar guides 21g are fixed at a predetermined distance from each other in the central portion in the width direction of the upper frame 21b. A reclining device 60 is attached to the connecting portion of the cushion frame 11 and the back frame 21, and a slide rail device 40 is attached to the lower portion of the cushion frame 11 via a height device 50.

[0024] As shown in FIGS. 1 and 2, the headrest 30 is a head portion that supports the head of an occupant from behind, and includes a pillar 31 that serves as a core material, a pad material 32, and a cover material 33. The pillar 31 is held by left and right pillar guides 21g. An imaging device 70 (imaging camera) is fixed to the rear surface of the headrest 30 (pillar 31).

[0025] As shown in Figs. 1 and 2, the slide rail device 40 is attached between the seat cushion 10 and the vehicle floor V1 in the vertical direction. The slide rail device 40 is fixed to the vehicle floor V1 and comprises left and right lower rails 41 extending in the fore-and-aft direction of the seat, left and right upper rails 42 supported slidably along the lower rails 41, a locking member 43 that locks the upper rails 42 against the lower rails 41 so that they cannot slide, and a rail operating lever (not shown) that releases the locked state of the locking member 43. The seat cushion 10 (cushion frame 11) is attached to the upper surfaces of the left and right upper rails 42 via a height device 50.

[0026] The height device 50 is attached between the slide rail device 40 and the seat cushion 10, as shown in FIGS. The height device 50 includes left and right first links 51 arranged on the front side of the seat, left and right second links 52 arranged on the rear side of the seat, a locking member 53 that locks the seat cushion 10 so that it cannot be raised or lowered, and a height operating lever (not shown) that releases the locked state of the locking member 53. The height device 50 can adjust the height of the seat cushion 10 by using the second link 52 as a driving link, for example.

[0027] As shown in Figures 1 and 2, the reclining device 60 comprises a reclining main body 61 that is driven when rotating the seat back 20, a back rotation shaft 62, a spiral spring 63 that urges the seat back 20 to rotate forward around the back rotation shaft 62, and a reclining operation lever (not shown) that releases the locked state of the seat back 20.

[0028] The reclining body 61 has a known locking mechanism, and the state of the seat back 20 can be switched between a "locked state" in which it is fixed to the seat cushion 10, and an "unlocked state" in which it can rotate relative to the seat cushion 10. The back rotation shaft 62 is supported in the seat width direction on the seat cushion 10 side and the seat back 20 side. One end of a spiral spring 63 is anchored on the seat cushion 10 side, and the other end is anchored on the seat back 20 side.

[0029] The imaging device 70 is a small camera (wide-angle camera) that captures an image of an occupant (the occupant's face) seated in the vehicle seat 1, as shown in FIGS. The imaging device 70 captures an image of the occupant, generates “imaging data”, and transmits the “imaging data” to the control device 100. The imaging device 70 may be an imaging camera equipped with AI (artificial intelligence) and capable of analyzing captured images. The imaging device 70 may also be a 360-degree omnidirectional camera (a camera equipped with a fisheye lens), etc.

[0030] The imaging devices 70 are disposed at positions corresponding to the front seat 1A, the front seat 1B, the rear seat 1C, and the rear seat 1D, respectively. Specifically, the imaging devices 70A and 70B are attached to the ceiling V2 and disposed in front of the front seats 1A and 1B, respectively. The imaging devices 70C and 70D are attached to the rear surfaces of the front seats 1A and 1B, respectively, and disposed in front of the rear seats 1C and 1D. More specifically, the imaging device 70C (70D) is attached to the rear surface of the headrest 30 of the front seat 1A (1B) via a cover member . It is possible to appropriately change the number and mounting positions of the imaging devices 70. For example, as shown in FIG.

[0031] FIG. 3 is a rear view of the vehicle seat 1 of the modified example. In addition to the imaging device 70, imaging devices 71 to 76 are attached to the rear surface of the vehicle seat 1. The imaging device 71 is attached to the upper end of the headrest 30 via a cover member 34, and is disposed at a position higher than the imaging device 70. The imaging devices 72 are attached to the rear surface of the upper frame 21b, and are exposed rearward from the vehicle seat 1. The imaging devices 73 are attached to the left and right side portions of the upper frame 21b, and are exposed rearward from the vehicle seat 1. The imaging device 74 is attached to the center of the connecting frame 21d, and is exposed rearward from the vehicle seat 1. The imaging devices 75 are attached to the left and right side frames 21a, respectively, and are exposed rearward from the vehicle seat 1. The imaging device 76 is attached to the rear surface of the support plate 21f, and is exposed rearward from the vehicle seat 1. The imaging devices 72 to 76 penetrate the padding material 32 and the covering material 33 and are exposed rearward, and can capture images of the faces of occupants positioned rearward of the vehicle seat 1 (front seats 1A, 1B).

[0032] As described above, by attaching the imaging device 70 to the seat back 20 (the back frame 21) or the headrest 30 (the cover member 34), the imaging device 70 can be arranged inside the vehicle V in a compact manner. It is preferable to provide a vibration suppression device for suppressing vibration of the imaging device 70 in the portion of the vehicle seat 1 where the imaging device 70 is attached or in the surrounding area thereof. This can suppress blurring of the image captured by the imaging device 70. The vibration suppression device may be, for example, a gimbal device or an elastic member. The elastic member may be, for example, a spring or a vibration-damping sponge (soft foam having low resilience). Alternatively, the variable damping type damper may be one that utilizes air pressure, hydraulic pressure, or an oil syringe (viscous material).

[0033] The vehicle seat 1 may further include a switching device that switches the imaging device 70 from an exposed state to a stored state inside the vehicle seat 1 when a collision of the vehicle V is detected.

[0034] The display device 80 is a display used by a passenger seated in the vehicle seat 1, as shown in FIG. As shown in Figs. 5 to 8, the display device 80 displays, to the occupant, display information (first display information) relating to the facial expression of the occupant and display information (second display information) relating to operation information of the movable device. The display devices 80 are disposed at positions corresponding to the front seat 1A, the front seat 1B, the rear seat 1C, and the rear seat 1D, respectively. Specifically, the display devices 80A and 80B are attached to the ceiling V2 and disposed in front of the front seats 1A and 1B, respectively. The imaging devices 80C and 80D are attached to the rear surfaces of the seat backs 20 of the front seats 1A and 1B, respectively, and disposed in front of the rear seats 1C and 1D (the display devices are omitted in FIG. 2).

[0035] As shown in FIG. 1, the voice acquisition device 85 is a device that acquires the voice of an occupant seated in the vehicle seat 1, and is, for example, a microphone, a voice recorder, or the like. The voice acquisition device 85 acquires the voice of the occupant, generates “voice data”, and transmits the “voice data” to the control device 100 . The voice acquisition devices 85 are disposed at positions corresponding to the front seat 1A, the front seat 1B, the rear seat 1C, and the rear seat 1D. The voice acquisition device 85 may be mounted inside the display device 80. In other words, the voice acquisition device 85 may be configured as a part of the display device 80.

[0036] The vibration device 90 is also called a stimulating device, and as shown in FIG. 1, is a device that applies vibration (stimulation) to an occupant seated in the vehicle seat 1, and is, for example, a vibration motor (unbalanced mass motor). The vibration device 90 is disposed on the surface of the seat cushion 10, and operates based on a control signal received from the control device 100 to apply vibrations to the occupant. The vibration devices 90 are disposed at positions corresponding to the front seat 1A, the front seat 1B, the rear seat 1C, and the rear seat 1D.

[0037] The collision detection sensor 95 is also called a collision sensor, and as shown in FIG. 1, is a sensor that detects a collision (precursor to a collision) of the vehicle V, and is, for example, an acceleration sensor. The collision detection sensor 95 is mounted inside the vehicle V, and transmits a detection signal to the control device 100 when it detects a collision of the vehicle V (a precursor to a collision).

[0038] The vehicle seat system S may further include a sound generating device (speaker) that generates a predetermined sound or music for an occupant seated in the vehicle seat 1. The vehicle seat system S may further include an execution sensor (execution switch) for executing image capture by the imaging device 70. In this case, the execution sensor may be disposed on an interior member of the vehicle or on the outer surface of the vehicle seat. For example, an occupant can press and operate the execution sensor to capture an image by the imaging device 70. At this time, a control process using the captured image data described below may be performed to select the occupant with the most smile among multiple occupants, and operate the vehicle seat 1 in which the occupant is seated. Alternatively, the smile of the occupant may be displayed on the display device 80.

[0039] The control device 100, as shown in Figs. 1 and 4, is a device that performs overall electrical control of the vehicle V, and is, for example, an engine control unit (ECU). The control device 100 is connected to the slide rail device 40, the height device 50, the reclining device 60, the imaging device 70, the display device 80, the voice acquisition device 85, the vibration device 90, and the collision detection sensor 95 via an in-vehicle network. Specifically, the control device 100 is a computer equipped with a CPU (processor) as a data arithmetic and control processing device, a ROM and RAM as storage devices, an HDD, and a communication interface for transmitting and receiving information data via an in-vehicle network.

[0040] In addition to a main program that performs the functions required of a computer, a vehicle seat program according to this embodiment is stored in the storage device of the control device 100. The functions of the vehicle seat system S are realized by executing the program by a CPU (processor).

[0041] <Functions of vehicle seat systems> As shown in FIG. 3, from a functional standpoint, the control device 100 mainly comprises a memory unit 110 for storing various programs and various data, a facial expression recognition unit 111, a facial expression determination unit 112, a calculation unit 113, a movable amount determination unit 114, a movement control unit 115, an output unit 116, and a physical information detection unit 117. These are composed of a CPU (processor), ROM, RAM, HDD, communication interface, and various programs.

[0042] The storage unit 110 stores "imaging data," "audio data," "occupant physique data," "movement amount data," etc. "Imaging data" and "occupant physique data" are data obtained from the imaging device 70, and "audio data" is data obtained from the audio acquisition device 85, and are stored in a centralized manner. "Movement amount data" is data indicating the movement amount of the movable devices 40, 50, 60, and is stored in a centralized manner. By referring to these data, the control device 100 can move the movable devices 40, 50, 60 in response to the facial expression, voice, and physique of each occupant. In addition, information related to the facial expression of the occupant and information related to the operation information of the movable devices can be output to the display device 80. Storage unit 110 also stores criteria data such as "facial expression criteria data" and "degree of change criteria data" which will be described later.

[0043] The facial expression recognition unit 111 recognizes the facial expression of an occupant seated in the vehicle seat 1. Specifically, the facial expression recognition unit 111 acquires "imaging data" captured by the imaging device 70, recognizes the facial expression of the occupant based on the "imaging data", and acquires "recognition information relating to facial expression". More specifically, the facial expression recognition unit 111 detects position information of the components of the passenger's face and angle information indicating the tilt angle of the head based on the "imaging data." Then, it obtains recognition information on the facial expression based on the position information and angle information.

[0044] The recognition information includes, for example, information on the movement of lowering eyebrows, lifting outer eyebrows, lifting cheeks, lifting chin, making dimples, closing eyes, opening eyes, lifting inner eyebrows, lowering chin and opening lips, tensing eyelids, lifting both ends of lips, pressing lips, pursing lips, pulling both ends of lips sideways, sucking in lips, opening mouth, wrinkling nose, lifting upper lip, blinking, and up-down, left-right, and tilting head.

[0045] The facial expression determination unit 112 determines the type of facial expression of the occupant (for example, whether the occupant is smiling) based on the recognition information by the facial expression recognition unit 111. Specifically, the facial expression determination unit 112 determines the type of facial expression of the occupant and the degree of change in the facial expression (the magnitude of the facial expression) based on the above recognition information. For example, the facial expression determination unit 112 determines a "smile" as the predetermined facial expression of the occupant, and evaluates the "size of the smile" as the degree of change in the smile into multiple levels (multiple stages), and presents the evaluation level. A higher evaluation level indicates a greater change in facial expression. It is preferable that the storage unit 110 stores "facial expression reference data" for determining the type of facial expression of the occupant, and "degree of change reference data" for determining the degree of change in a predetermined facial expression.

[0046] The facial expression determination unit 112 may determine the type of facial expression of the occupant as a "smile", or may determine the facial expression of the occupant as a surprised face, a happy face, a fearful face, a sad face, an angry face, a contemptuous face, or a blank face.

[0047] The calculation unit 113 calculates the number of times a predetermined facial expression has been determined by the facial expression determination unit 112 . Specifically, the calculation unit 113 calculates the number of times a predetermined facial expression (for example, a smile) is determined by the facial expression determination unit 112 within a predetermined time period. The calculation unit 113 determines that a smile has occurred once if the smile exceeds a first reference time. At this time, the calculation unit 113 may determine that a smile has occurred twice if the smile continues to exceed a second reference time, or three times if the smile continues to exceed a third reference time. The calculation unit 113 may calculate the time during which a smiling expression is made (the time during which it is made continuously) within a predetermined period of time.

[0048] The movement amount determination unit 114 determines the amount of movement of the vehicle seat 1 (seat body 1a) based on the number of times the predetermined facial expression (smile) is determined as calculated by the calculation unit 113. Specifically, the movement amount determination unit 114 determines the movement amount of the movable device (the amount of rise of the height device 50) based on the number of times a smile is determined. The movement amount determination unit 114 may determine the movement amount of the height device 50 based on the number of times a smile is determined and the degree (magnitude) of change in the smile. For example, the movement amount determination unit 114 may determine a larger movement amount (1.5 times or 2 times the movement amount) when the degree of the smile is large when the smile is determined (when the evaluation level is higher than normal). Alternatively, the movement amount determination unit 114 may determine a smaller movement amount (0.5 times the movement amount) when the degree of the smile is small when the smile is determined (when the evaluation level is lower than normal).

[0049] The movement control unit 115 controls the movement of the movable device based on the determination information by the facial expression determination unit 112 . Specifically, the operation control unit 115 controls the operation of the movable devices 40, 50, 60 based on the amount of movement of the seat body determined based on the determination information by the facial expression determination unit 112 and the information of the calculation result by the calculation unit 113. In other words, the operation control unit 115 moves a predetermined movable device based on the amount of movement determined by the amount of movement determination unit 114. The operation control unit 115 may control the seat body to return to the "reference position" after a predetermined time has elapsed after moving a part or the whole of the seat body from the "reference position (position where an occupant can sit)" to the "first movement position". In other words, the operation control unit 115 may move the seat body continuously from the "first movement position" to the "second movement position", or may move the seat body from the "reference position" to the "second movement position".

[0050] An example of the control by the operation control section 115 will be described with reference to FIGS. The facial expression recognition unit 111 recognizes the facial expressions of a plurality of occupants. Then, the facial expression determination unit 112 determines the smile and the size of the smile for each passenger based on the above recognition information. Then, the operation control unit 115 raises the height device 50 so as to move the vehicle seat 1 in which a specific occupant is seated, based on the smile of each occupant and the size of the smile of each occupant.

[0051] 5, it can be seen that the height device 50 is not operating in the right front seat 1A, whereas it can be seen that the height device 50 is raising the seat body in the right rear seat 1C. At this time, it is preferable that a display screen shown in Fig. 6, which will be described later, is displayed on the display device 80. On the display screen, information related to the seat states of the vehicle seats 1A-1D and information related to the facial expressions (smile, degree of smile, number of smiles) of the occupants corresponding to the vehicle seats 1A-1D are displayed in conjunction with each other. In other words, when a passenger seated in the vehicle seat 1 smiles, the control device 100 controls the height device 50 to raise the seat body. This allows passengers to compete with each other, improving entertainment value.

[0052] The control by the operation control unit 115 may be executed as shown in FIG. The facial expression recognition unit 111 recognizes the facial expressions of a plurality of occupants. Then, the facial expression determination unit 112 determines the smile and the size of the smile for each passenger based on the above recognition information. Then, the operation control unit 115 raises the height device 50 and the reclining device 60 so as to move the vehicle seat 1 in which a specific occupant is seated, based on the smile of each occupant and the size of the smile (number of times the occupant smiles) of each occupant.

[0053] 7, it can be seen that in the right front seat 1A, the height device 50 is raising the seat body, and in the right rear seat 1C, the height device 50 is raising the seat body, and the reclining device 60 is moving the seat body rearward. At this time, it is preferable that the display device 80 displays a display screen as shown in Fig. 8. On the display screen, information related to the seat states of the vehicle seats 1A-1D and information related to the facial expressions (smile, size of smile, number of times of smile) of the occupants corresponding to the vehicle seats 1A-1D are displayed. In other words, when the passenger seated in the vehicle seat 1 smiles, the height adjustment device 50 and the reclining device 60 are controlled to move. This can improve entertainment value.

[0054] Furthermore, the operation control unit 115 acquires "occupant's voice data" obtained by the voice acquisition device 85, and controls the operation of the vibration device 90 based on the volume of the occupant's voice. By doing so, it can be recognized that the occupant is laughing out loud by the vibration operation of the vibration device 90. At this time, the vibration device 90 of the vehicle seat 1 in which the relevant occupant is seated may be operated, or the vibration devices 90 of multiple vehicle seats 1 may be operated. In this way, it is possible to increase the entertainment value inside the vehicle.

[0055] In addition, the operation control unit 115 may control the movable device (height device 50) to move the seat body based on the occupant's smile determined by the facial expression determination unit 112 and the volume of the occupant's voice obtained by the voice acquisition device 85. By doing so, it is possible to more accurately determine whether the occupant is laughing, and to operate the movable device based on the determination result. Specifically, the following applies: The movement amount determination unit 114 multiplies the movement amount of the movable device by a predetermined coefficient based on the voice data (volume of voice) obtained by the voice acquisition device 85. For example, the movement amount determination unit 114 increases the movement amount when the voice of the occupant is louder than a predetermined voice reference value, and decreases the movement amount when the voice is quieter than the voice reference value. Alternatively, the movement amount determination unit 114 (which may be called the laughter determination unit) determines whether the occupant's voice is a "normal voice" or a "laughing voice", and if it determines that the voice is a "laughing voice", it multiplies the movement amount of the movable device (reference movement amount) by a predetermined coefficient (laughing voice coefficient). When determining whether the voice is a "laughing voice", the movement amount determination unit 114 may make the determination based on the voice waveform data. Note that if the voice waveform is interrupted, it may be determined to be a "fake laugh" rather than a "laughing voice", and the multiplication by the laughter coefficient may not be performed. Then, the movement control unit 115 moves the movable devices 40 , 50 , 60 based on the movement amount determined by the movement amount determination unit 114 .

[0056] In addition, in the above configuration, when a collision of the vehicle V is detected by the collision detection sensor 95, the operation control unit 115 controls the movable devices 40, 50, 60 not to operate, regardless of the judgment information by the facial expression judgment unit 112 (the amount of movement determined by the amount of movement determination unit 114). In this way, the operation of the movable devices 40, 50, 60 can be stopped if a collision of the vehicle V is predicted or if the vehicle V has collided.

[0057] The output unit 116 outputs first display information in which the facial expression of the occupant determined by the facial expression determination unit 112 is converted into a facial expression of a character, and second display information based on the operation information of the movable device controlled by the operation control unit 115. The first display information is, for example, as shown in Fig. 6 and Fig. 8. The facial expression of each passenger is converted into the facial expression of a character (face mark) and displayed. Specifically, it can be seen that the passenger seated in the right rear seat 1C is smiling. The second display information is, for example, as shown in FIG. 6 (FIG. 8). The number of times a smile has been determined for each occupant is counted and displayed. Specifically, it can be seen that the number of times a smile has been determined for the occupant seated in the right rear seat 1C is three (four) times. Since the amount of movement of the movable device is determined based on the number of times a smile has been determined, this is equivalent to information based on the operation of the movable device being displayed. In other words, it can be seen that the right rear seat 1C has a larger amount of movement than the right front seat 1A and the left front seat 1B.

[0058] More specifically, the output unit 116 outputs display information in which the facial expression of the character is changed based on the change in the facial expression of the passenger determined by the facial expression determination unit 112 . In this case, the character may be a facial mark, an avatar that pays homage to the face of a passenger, or an animal. The number of times a smile has been determined may be displayed by the number of cushions, or by the character's movements (the character gradually standing up).

[0059] The physique information detection unit 117 detects physique information of the occupant seated in the vehicle seat 1. For example, the physique information detection unit 117 measures the length of the upper body, the length of the thighs, and the length of the lower legs as physique information of the occupant based on the imaging data of the occupant, and then mainly detects the position of the head of the occupant.

[0060] By detecting the occupant's physical information, the following control can be performed. The operation control unit 115 controls the operation of the movable devices 40, 50, 60 based on the physique information of the occupant obtained by the physique information detection unit 117, and moves the seat body to a seating position that corresponds to the imaging range of the imaging device 70. This makes it easier for the imaging device 70 to recognize the facial expression (smile) of the occupant. In addition, the operation control unit 115 controls the height device 50 to limit the lifting and lowering operation of the seat body based on the physique information of the occupant. In this way, the operation of the height device 50 can be limited based on the physique information of the occupant, allowing the occupant to be seated comfortably.

[0061] More specifically, it is advisable to control as follows. When the physique information detection unit 117 detects the position of the occupant's head, the operation control unit 115 limits the rise of the height device 50 so that the top of the occupant's head falls within the imaging range. Furthermore, when the physique information detection unit 117 detects the position of the occupant's head, the operation control unit 115 operates the reclining device 60 so that the occupant's facial expression is easily detected by the imaging device 70 (front position). In addition, when the physique information detection unit 117 detects the position of the occupant's head, the operation control unit 115 limits the ascent of the height device 50 based on the occupant's physique so that the head does not come into contact with the ceiling V2 of the vehicle V. In addition, when the physique information detection unit 117 detects the position of the head of the occupant, the operation control unit 115 determines the amount of operation based on the physique information of the occupant so as not to cause discomfort to the occupant seated in the rear, and moves the movable devices 40, 50, 60.

[0062] Also, as shown in FIG. 9, after the physique information detection unit 117 detects the physique information of the occupant, the operation control unit 115 controls the height device 50 so that the occupant's head falls within the imaging range, and adjusts the height of the seat body. When the imaging device 70 moves in accordance with the operation of the reclining device 60 of the vehicle seat 1A, the operation control unit 115 may control the height device 50 of the vehicle seat 1C on which the occupant sits in response to the movement, and further adjust the height of the seat body. In addition to the height device 50, the slide rail device 40 and the reclining device 60 may also be operated together. This makes it easier to recognize the facial expression (smile) of the occupant.

[0063] Additionally, when the vehicle seat system S includes a plurality of image capture devices 70 to 76 as in the modified example shown in FIG. 3, it is preferable to control as follows. When the physique information detection unit 117 detects the position of the occupant's head, if it is difficult for the facial expression recognition unit 111 to recognize the facial expression of the occupant from the imaging data captured by the imaging device 70, the facial expression recognition unit 111 may recognize the facial expression of the occupant from the imaging data captured by another imaging device 71. In other words, the imaging devices 70 to 76 may be appropriately switched and used.

[0064] <How to move a vehicle seat> Next, the processing of a vehicle seat program (vehicle seat moving method) executed by the vehicle seat system S will be described with reference to FIG. The above-mentioned program of the present embodiment is a program for realizing the above-mentioned facial expression recognition unit 111, facial expression determination unit 112, calculation unit 113, movable amount determination unit 114, motion control unit 115, output unit 116, and physique information detection unit 117 as functional components of control device 100 having storage unit 110. The CPU (processor) of control device 100 executes this vehicle seat program. The execution of the above program is triggered by, for example, receiving an instruction to start the above system S through an operation by a passenger (user).

[0065] The vehicle seat movement flow shown in FIG. 10 starts with step 1 (S1) in which the facial expression recognition unit 111 recognizes the facial expression of an occupant seated in the vehicle seat 1. Specifically, the facial expression recognition unit 111 acquires "imaging data" captured by the imaging device 70, recognizes the facial expression of the occupant based on the "imaging data", and acquires "recognition information relating to facial expression".

[0066] Then, in step 2, the facial expression determination unit 112 determines the type of facial expression of the occupant (for example, whether the occupant is smiling) based on the recognition information by the facial expression recognition unit 111. Specifically, the facial expression determination unit 112 determines the type of facial expression of the occupant and the magnitude of the facial expression based on the above recognition information.

[0067] Then, in step 3, calculation unit 113 calculates the number of times facial expression determination unit 112 has determined that a smile has occurred. Specifically, the calculation unit 113 calculates the number of times a smile has been determined by the facial expression determination unit 112 within a predetermined period of time.

[0068] Then, in step 4, the movement amount determination unit 114 determines the movement amount of the vehicle seat 1 based on the number of times that a smile has been determined as calculated by the calculation unit 113. Specifically, the movement amount determination unit 114 determines the movement amount of the movable device based on the number of times a smile is determined. As described above, the movement amount determination unit 114 may determine the movement amount of the movable device based on the number of times a smile has been determined and the size of the smile.

[0069] Then, in step 5, the operation control section 115 moves a predetermined movable device based on the movement amount determined by the movement amount determination section 114 (see FIG. 5). That is, the operation control unit 115 controls the operation of the movable devices 40 , 50 , 60 based on the amount of movement of the seat body determined based on the determination information by the facial expression determination unit 112 and the information on the calculation result by the calculation unit 113 .

[0070] Then, in step 6, the output unit 116 outputs first display information in which the facial expression of the occupant determined by the facial expression determination unit 112 is converted into a facial expression of the character, and second display information based on the operation information of the movable device controlled by the operation control unit 115. Note that the second display information can also be said to be information based on the number of times a smile has been determined and the size of the smile. Specifically, it is as shown in the display screen of FIG.

[0071] Then, in step 7, the control device 100 judges whether or not a command to stop the vehicle seat system S has been received by an operation of the occupant. If a command to stop the system S has been received (step 7: Yes), the process of Fig. 10 ends. On the other hand, if a command to stop the system S has not been received (step 7: No), the process returns to step 1 and the above process is repeated.

[0072] The vehicle seat system and vehicle seat program described above make it possible to control the operation of the movable device based on the biological information of the occupant (e.g., the facial expression of the occupant), and also makes it possible for the occupant seated in the vehicle seat to feel comfortable inside the vehicle.

[0073] <Other embodiments> In addition to the above embodiments, the vehicle seat system S may be configured to simply control the recognition of the facial expression of an occupant seated in the vehicle seat 1, and when a predetermined facial expression (the occupant's smile) is determined, control is performed to operate a predetermined movable device (height device 50). Alternatively, the vehicle seat system S may simply control the vehicle seat 1 by recognizing the facial expression of an occupant seated in the vehicle seat 1 to determine whether the occupant is smiling, and when the occupant's voice is recognized, control the height device 50 to operate.

[0074] In addition to the above embodiment, the vehicle seat system S may be configured to include a vehicle seat 1, an imaging device 70, a voice acquisition device 85, a lighting device that adjusts the brightness inside the vehicle, and a control device 100 that controls these devices. The control device 100 may change the illumination color of the lighting device according to the volume of the passenger's voice acquired by the voice acquisition device 85, and execute imaging by the imaging device 70. This allows passengers to be photographed with natural expressions and also reduces energy consumption due to lighting. The system S may determine the passenger who is the loudest among multiple passengers, adjust the lighting at the seat where the passenger is seated, and capture the image. This allows the lighting to highlight the passenger who talks a lot, and allows the passenger's natural facial expression to be captured. The system S may further include a voice recording device (voice recording function) for recording the voices of the passengers. In this way, image data and voice data can be obtained simultaneously, improving entertainment value. The system S may control the operation of the imaging device 70 attached to the rear surface of the seat back 20 based on the posture of the seat body (seat back 20). By doing so, the angle of view of the imaging device 70 (imaging camera) can be appropriately maintained in accordance with the posture of the seat body. In the above system S, when the driving operation part (operating handle) of the vehicle V is attached to the rear surface of the vehicle seat 1, it is preferable that the imaging device 70 is disposed at a position on the rear surface that does not overlap with the driving operation part. In this way, it is possible to capture images without interfering with the driving operation part.

[0075] In addition to the above embodiment, in the vehicle seat system S, the imaging device 70 may be attached to the rear surface of the vehicle seat 1 and to the outer surface of the vehicle interior material (vehicle door). By doing so, it is possible to select an imaging device 70 suitable for imaging (an imaging device 70 that is not in a backlit position). In other words, it is no longer necessary to adjust the brightness of the lighting inside the vehicle each time. There is also no need to operate the blinds. In the above system S, the control device 100 may control the movable device to move the vehicle seat 1 to a position suitable for imaging.

[0076] In the above embodiment, a vehicle seat system for use in an automobile has been described as a specific example, but without being limited thereto, the vehicle seat system can be used in vehicles such as trains, buses, etc., as well as airplanes, ships, etc.

[0077] In the above embodiment, a vehicle seat system according to the present invention has been mainly described. However, the above embodiment is merely an example for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be modified or improved without departing from the spirit of the present invention, and the present invention naturally includes equivalents thereof. [Explanation of symbols]

[0078] V Vehicles V1 Vehicle Floor V2 Ceiling S Vehicle seat system 1(1A~1D) Vehicle seats 1A Front seat (driver's seat) 1B Front seat (passenger seat) 1C, 1D rear seats 1a Seat body 10 Seat cushion 11 Cushion frame 11a Side frame 11b Pan Frame 11c Rear-linked frame 11d Elastic spring 12 Pad material 13 Skin material 20 Seat back 21 Back Frame 21a Side Frame 21b Upper frame 21c Lower Frame 21d Joint Frame 21e Wire material 21f Support plate 21g Pillar Guide 22 Pad material 23 Skin material 30 Headrest 31 Pillar 32 Pad material 33 Skin material 34 Cover member 40 Slide rail device (movable device) 41 Lower rail 42 Upper Rail 43 Locking member 50 Height device (movable device) 51 First link 52 Second Link 53 Locking member 60 Reclining device (movable device) 61 Reclining unit 62 Back rotation shaft 63 Spiral Spring 70 (70A to 70D) Imaging device 71~76 Imaging device 80(80A~80D) Display device 85 Audio capture device 90 Vibration device 95 Collision detection sensor (collision detection section) 100 Control device 110 Storage section 111 Facial expression recognition unit 112 Facial expression determination section 113 Arithmetic section 114 Movement amount determining section 115 Motion control section 116 Output section 117 Physical information detection unit

Claims

1. A vehicle seat including a seat body on which an occupant can sit and a movable device that moves a part or all of the seat body; A control device that controls an operation of the movable device, The control device includes: a facial expression recognition unit that recognizes a facial expression of an occupant seated in the vehicle seat; an expression determination unit that determines a type of facial expression of the occupant based on the recognition information by the facial expression recognition unit; A vehicle seat system comprising: an operation control unit that controls the operation of the movable device based on determination information by the facial expression determination unit.

2. The facial expression recognition unit recognizes facial expressions of a plurality of the occupants, The facial expression determination unit determines a degree of change in a facial expression of each of the occupants based on the recognition information for each of the occupants, The vehicle seat system according to claim 1, characterized in that the operation control unit controls the movable device to move the seat body in which a specific occupant is seated, based on the type of facial expression for each occupant determined by the facial expression determination unit and the degree of change in the facial expression.

3. an imaging device for imaging an occupant seated in the vehicle seat, The control device has a physique information detection unit that detects physique information of the occupant, The operation control unit is controlling an operation of the movable device based on the physique information of the occupant detected by the physique information detection unit; 2. The vehicle seat system according to claim 1, wherein the seat body is moved to a seating position corresponding to an imaging range of the imaging device.

4. The movable device is a height device that connects the seat body to a vehicle floor so that the seat body can be raised and lowered, the control device has a calculation unit that calculates a number of times a predetermined facial expression has been determined by the facial expression determination unit, The vehicle seat system according to any one of claims 1 to 3, characterized in that the operation control unit controls the height device to raise the seat body based on the number of times the specified facial expression is determined by the calculation unit.

5. a display device that is used by a passenger seated in the vehicle seat, the control device has an output unit that outputs first display information obtained by converting the facial expression of the occupant determined by the facial expression determination unit into a facial expression of a character, and second display information based on operation information of the movable device controlled by the operation control unit, The vehicle seat system according to any one of claims 1 to 3, characterized in that the display device displays the first display information and the second display information output by the output unit toward the occupant.

6. a display device that is used by a passenger seated in the vehicle seat, The control device includes: a calculation unit that calculates the number of times a predetermined facial expression is determined by the facial expression determination unit; an output unit that outputs display information based on the number of times the predetermined facial expression has been determined, the display information being calculated by the calculation unit; 4. The vehicle seat system according to claim 1, wherein the display device displays the display information output by the output unit toward the occupant.

7. The control device includes: a calculation unit that calculates the number of times a predetermined facial expression is determined by the facial expression determination unit; a movement amount determination unit that determines a movement amount of the seat body based on the number of times the predetermined facial expression is determined, the movement amount being determined by the calculation unit; The vehicle seat system according to any one of claims 1 to 3, characterized in that the operation control unit controls the movable device based on the amount of movement of the seat body determined by the movement amount determination unit, thereby moving the seat body.

8. a voice acquisition device for acquiring a voice of an occupant seated in the vehicle seat, The vehicle seat system according to any one of claims 1 to 3, characterized in that the operation control unit controls the movable device to move the seat body based on the smiling facial expression of the occupant determined by the facial expression determination unit and the degree of the occupant's voice obtained by the voice acquisition device.

9. a voice acquisition device for acquiring a voice of an occupant seated in the vehicle seat; a vibration device that applies vibration to an occupant seated in the vehicle seat, The vehicle seat system according to any one of claims 1 to 3, wherein the operation control unit controls the operation of the vibration device based on the level of the occupant's voice obtained by the voice acquisition device.

10. a collision detection unit that detects a collision of the vehicle; The vehicle seat system according to any one of claims 1 to 3, characterized in that when the collision detection unit detects a collision of the vehicle, the operation control unit controls the movable device not to operate regardless of the judgment information by the facial expression judgment unit.

11. The movable device is a height device that connects the seat body to a vehicle floor so that the seat body can be raised and lowered, the control device has a physique information detection unit that detects physique information of an occupant seated in the vehicle seat, The vehicle seat system according to any one of claims 1 to 3, characterized in that the operation control unit controls the height device so as to limit the raising and lowering movement of the seat body based on physique information of the occupant.

Images