System for recognizing gestures of vehicle occupant

Abstract

A passenger gesture recognition system for gesture recognition of a passenger of a vehicle is performed using passenger gestures limited to seatbelt surroundings. The system includes a controller having: a target area detection unit that detects a detection target area around a strap of a seatbelt tied to a passenger based on an image captured by an image sensor; a shielded region detection unit that detects a region shielded by the gesture of the passenger within the detection target region; and a signal output unit that outputs an output signal specified for the obscured area and controls execution of a function regarding the output signal.

Description

Technical Field

[0001] The present disclosure relates to a passenger gesture recognition system for a vehicle, and more particularly, to a passenger gesture recognition system for performing and recognizing gestures in an area around a seat belt of the vehicle. Background Art

[0002] Recently, various systems have been proposed to recognize passenger gestures in vehicles.

[0003] However, existing systems have problems, including complex algorithms and high development costs (e.g., labor costs for annotation of many databases).

[0004] In addition, in addition to the disadvantage of increased material costs due to the use of high-performance hardware (image sensor, 3D camera, TOF camera, high-performance processor, etc.), there is also a disadvantage of camera recognition errors caused by light reflection and light blur due to external conditions.

[0005] Specifically, there is an inconvenience in that an area where a passenger makes a gesture is designated, and thus the user must move within the designated area.

[0006] For example, in the case of a relaxation gesture (eg, leaning back on a vehicle seat back), it may be difficult to recognize the gesture because the distance from the designated area increases.

[0007] Furthermore, if the gesture is large and inaccurate, it is difficult to correctly recognize the motion, which increases the inconvenience compared to directly pressing the button.

[0008] Furthermore, because gestures and functions are not intuitive, it is difficult to use the functions, and the types of gestures and functions that can be connected are limited.

[0009] In addition, a method of improving the gesture recognition rate by wearing a wrist-worn wearable device has been proposed, but it is inconvenient for the user to wear the wearable device, such an additional device incurs costs, and there is also a risk of losing the device.

[0010] The matters described above as background art are only for the purpose of enhancing understanding of the background of the present disclosure and should not be regarded as an admission that they correspond to the prior art known to those skilled in the art. Summary of the invention

[0011] The present disclosure provides a passenger gesture recognition system for a vehicle, which is capable of simplifying an algorithm for gesture recognition and improving a gesture recognition rate by executing and recognizing gestures limited to a peripheral area of ​​a seat belt.

[0012] According to one aspect of the present disclosure, the above and other purposes can be achieved by providing a system for recognizing a passenger's gesture, the system comprising: a target area detection unit, configured to: detect a detection target area based on an image captured by an image sensor, the detection target area being located around a belt of a seat belt fastened to the passenger; an obscured area detection unit, configured to: detect an area within the detection target area that is obscured by the passenger's gesture; and a signal output unit, configured to: output an output signal specified for the obscured area, and control the execution of a function regarding the output signal.

[0013] The target area detection unit may detect a pattern within the detection target area.

[0014] The pattern may be printed on a belt cover provided on the belt, or may be printed on the belt itself.

[0015] The target area detection unit may detect a belt cover provided on the belt based on the captured image of the passenger, set a cover area including the belt cover as the detection target area, and detect a pattern provided on the belt cover within the set cover area.

[0016] The target area detection unit may detect an upper body of the passenger based on the captured image of the passenger, set the upper body area as the detection target area, and detect the pattern provided on the belt in the set upper body area.

[0017] The blocked area may be an area of ​​the pattern where the pattern blocked by the gesture is located.

[0018] The signal output unit may output an output signal designated for the mask pattern.

[0019] The target region detection unit may detect a belt cover provided on the belt based on the captured image of the passenger, set a cover region including the belt cover as the detection target region, divide the set cover region, and detect the divided region.

[0020] The system may further include a hand detection unit configured to detect a hand of a passenger located within the detection target area through a hand gesture, and the blocked area may be a segmented area blocked by the hand of the passenger.

[0021] The signal output unit may output an output signal designated for masking the segmented area.

[0022] According to another aspect of the present disclosure, a system for recognizing a passenger's gesture is provided, the system comprising: a target area detection unit configured to detect a detection target area based on an image captured by an image sensor, the detection target area being located around a belt of a seat belt fastened to the passenger; a hand detection unit configured to detect a hand of the passenger located within the detection target area through a gesture when the passenger makes a gesture; and a signal output unit configured to output an output signal specified for the location of the hand and control execution of a function regarding the output signal.

[0023] The target region detection unit may detect an upper body of the passenger and set the upper body region as the detection target region, and the hand detection unit may detect a position of the hand on a belt path located in the upper body region.

[0024] The signal output unit may obtain a classification value of the hand position by comparing the coordinates of the detected hand position with the designated coordinates of the upper body region, and output an output signal designated for the obtained classification value.

[0025] The vehicle may include a system for recognizing passenger gestures.

[0026] According to another aspect of the present disclosure, a system for recognizing a passenger's gesture is provided, the system comprising: a gesture tracking unit configured to track the gesture movement within a detection target area based on an image captured by an image sensor when the passenger performs a gesture of moving along a belt of a seat belt fastened to the passenger; and a signal output unit configured to output an output signal specified for the gesture movement and control execution of a function regarding the output signal.

[0027] The system may further include: a target area detection unit configured to detect a detection target area around the belt; and an object detection unit configured to detect a gesture object located within the detection target area through a gesture, and the gesture tracking unit may track the gesture object moving within the detection target area through the passenger's gesture.

[0028] The target area detection unit may detect the belt of the seat belt based on the captured image of the passenger and set the belt as the detection target area, and the object detection unit may detect one of the passenger's hand, belt cover and belt clip moving along the belt as a gesture object.

[0029] The surface of the gesture object may be coated with a reflective paint.

[0030] The signal output unit may determine whether there is an output signal beam designated for the detected hand shape.

[0031] The system may further include a gesture sensor configured to be movable along the belt by a gesture of the passenger, and the gesture tracking unit may track the gesture sensor moving along the belt when the passenger makes a gesture.

[0032] The signal output unit may determine whether there is an output signal beam designated for the movement pattern of the gesture sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other objects, features and other advantages of the present disclosure will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings, in which:

[0034] Figure 1 is a diagram showing the configuration of first and second embodiments of a gesture recognition system according to the present disclosure;

[0035] Figure 2 is a diagram showing a configuration of a third embodiment of a gesture recognition system according to the present disclosure;

[0036] Figure 3 is a diagram showing a configuration of a fourth embodiment of a gesture recognition system according to the present disclosure;

[0037] Figure 4 is a diagram showing a configuration of a fifth embodiment of a gesture recognition system according to the present disclosure;

[0038] Figure 5 is a diagram showing a configuration of a sixth embodiment of a gesture recognition system according to the present disclosure;

[0039] Figure 6 is a diagram showing a location where an image capturing component is installed according to the present disclosure;

[0040] Figures 7 to 9 is a diagram showing a pattern printed on a belt cover according to the present disclosure;

[0041] Figures 10 to 12 is a diagram for describing an operation of recognizing a gesture according to a first embodiment of the gesture recognition system of the present disclosure;

[0042] Figures 13 to 15 is a diagram for describing an operation of recognizing a gesture according to a second embodiment of the gesture recognition system of the present disclosure;

[0043] Figures 16 to 18 is a diagram for describing an operation of recognizing a gesture according to a third embodiment of the gesture recognition system of the present disclosure;

[0044] Figures 19 to 21 is a diagram for describing an operation of recognizing a gesture according to a fourth embodiment of the gesture recognition system of the present disclosure;

[0045] Figure 22 to Figure 24is a diagram for describing an operation of recognizing a gesture according to a fifth embodiment of the gesture recognition system of the present disclosure;

[0046] Figures 25 to 27 is a diagram showing different hand shapes used in a fifth embodiment of the gesture recognition system of the present disclosure;

[0047] Fig.28 is a flowchart showing a process of recognizing a gesture according to a first embodiment of the gesture recognition system of the present disclosure;

[0048] Fig.29 is a flowchart showing a process of recognizing a gesture according to a second embodiment of the gesture recognition system of the present disclosure;

[0049] Fig.30 is a flowchart showing a process of recognizing a gesture according to a third embodiment of the gesture recognition system of the present disclosure;

[0050] Fig.31 is a flowchart showing a process of recognizing a gesture according to a fourth embodiment of the gesture recognition system of the present disclosure;

[0051] Fig.32 is a flowchart showing a process of recognizing a gesture according to a fifth embodiment of the gesture recognition system of the present disclosure; and

[0052] Fig.33 Detailed description is a flowchart illustrating a process of recognizing a gesture according to a sixth embodiment of the gesture recognition system of the present disclosure. DETAILED DESCRIPTION

[0053] It should be understood that the term "vehicle" or "vehicular" or other similar terms used herein include general motor vehicles, such as passenger cars including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, watercraft including various boats and ships, aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles and other alternative fuel vehicles (e.g., fuels derived from resources other than petroleum). As mentioned herein, a hybrid vehicle is a vehicle having two or more power sources, such as gasoline-powered and electric vehicles.

[0054] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms "a / an" and "the" are also intended to include plural forms. It should also be understood that when used in this specification, the term "comprise and / or comprising" stipulates that there are stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or increase of one or more other features, integers, steps, operations, elements, parts and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. Throughout the specification, unless there is a clear description to the contrary, the word "comprise" and variants such as "comprises or comprising" will be understood to imply the inclusion of stated elements, but do not exclude any other elements. In addition, the terms "unit", "-part (-er)", "-device (-or)" and "module" described in the specification mean a unit for processing at least one function and operation, and can be implemented by hardware components or software components and combinations thereof.

[0055] In addition, the control logic of the present disclosure may be embodied as a non-transitory computer-readable medium on a computer-readable medium containing executable program instructions executed by a processor, controller, etc. Examples of computer-readable media include, but are not limited to, ROM, RAM, compact disk (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer-readable medium may also be distributed in a network-coupled computer system so that the computer-readable medium is stored and executed in a distributed manner, such as by a telematics server or a controller area network (CAN).

[0056] Reference will now be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or like parts.

[0057] In the following description of the embodiments disclosed in this specification, when the known functions and configurations incorporated herein may obscure the subject matter of the present disclosure, the detailed description thereof will be omitted. In addition, the accompanying drawings are provided only to facilitate the understanding of the embodiments disclosed in this specification, and do not limit the technical spirit disclosed herein, and include all changes, equivalents and substitutes included in the spirit and scope of the present disclosure.

[0058] The terms "first" and / or "second" are used to describe various components, but these components are not limited by these terms. The terms are used to distinguish one component from another component.

[0059] When one component is "coupled" or "connected" to another component, it should be understood that although the component may be directly coupled or connected to another component, a third component may exist between the two components. When one component is "directly coupled" or "directly connected" to another component, it should be understood that there is no element between the two components.

[0060] Unless the context clearly indicates otherwise, elements described in the singular are intended to include plural elements.

[0061] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0062] The configuration of the first to third embodiments of the passenger gesture recognition system according to the present disclosure includes: a target area detection unit 100, which detects a detection target area around a belt 500 fastened to a passenger's seat belt; a covered area detection unit 120, which detects an area covered by a gesture within the detection target area when the passenger makes a gesture; and a signal output unit 130, which outputs an output signal specified for the covered area and controls the execution of a function regarding the output signal.

[0063] For example, Figure 1 The depicted target area detection unit 100, the obscured area detection unit 120, and the signal output unit 130 may be executed by a controller. The controller may be provided with one or more modules and / or devices including the above-mentioned units. For example, the modules and / or devices may constitute hardware components forming part of the controller (e.g., modules or devices of a high-level controller), or may constitute separate controllers each having a processor and a memory. The vehicle according to the present disclosure may include one or more processors and a memory.

[0064] For example, the target area detection unit 100 detects the detection target area based on an image captured by the image sensor 400, and the image sensor 400 may be a visual sensor that detects infrared (IR) and RGB (color).

[0065] The image sensor 400 is mounted on Figure 6 The vehicle compartment is shown, and a passenger sitting on a seat and a seat belt belt 500 are detected.

[0066] At least one image sensor 400 may be installed. If one image sensor 400 is installed, it may be installed at the center of the front portion of the windshield or the roof, and if two image sensors are installed, they may be installed at both sides of the front portion of the windshield or the roof.

[0067] In addition, the target area detection unit 100 detects the detection target area by limiting the detection target area to an area around the seat belt belt 500. The belt 500 may be a shoulder belt worn on the shoulder and upper body of the passenger.

[0068] Therefore, hardware resources are saved by reducing the area in which the target area detection unit 100 operates to detect the detection target area, and misdetection and malfunction of the detection target area are prevented by preventing operation in an area without the tape 500 .

[0069] The occlusion region recognized by the occlusion region detection unit 120 is a portion of the detection target region occluded by a passenger's (user's) hand according to a gesture of the passenger, and the occlusion region may be confirmed based on an image captured by the image sensor 400 .

[0070] The signal output unit 130 is used to output an output signal regarding the portion blocked by the blocked area.

[0071] The signal output from the signal output unit 130 may be used to operate electronic devices mounted on the vehicle, such as an infotainment device, a navigation system, an air conditioning system, a seat device, and a driving assistance device.

[0072] That is, by specifying a detection target area limited to belt 500 of a seat belt that is always worn by a passenger, the algorithm is simplified and the labeled image capacity used for the DB is minimized, development costs are reduced, and the cost of hardware construction is minimized using an algorithm configuration that can operate adequately even with low-quality images.

[0073] Furthermore, since commands are made with a simple gesture covering the band 500, gestures can be made without being aware of a designated area where the gesture must be made, which not only improves the convenience of making gestures but also improves the gesture recognition rate.

[0074] In the present disclosure, the object area detection unit 100 may detect the pattern P within the detection object area.

[0075] For example, Figures 10 to 12 As shown, the pattern P may be printed on a tape cover 510 provided on the tape 500 .

[0076] That is, the designated pattern P may be printed on the belt cover 510 or the ring-shaped belt clip of the safety belt. The belt cover 510 can move along the belt 500 without interfering with the insertion / extraction of the belt 500.

[0077] As another example, Figures 13 to 15 As shown, the pattern P can be printed directly on the tape 500 .

[0078] refer to Figure 1 and Figures 10 to 12 The configuration of the first embodiment of the gesture recognition system is described in detail. The target area detection unit 100 can detect the belt cover 510 provided on the belt 500 based on the captured image of the passenger, set the cover area A1 including the belt cover 510 as the detection target area, and detect the pattern P provided on the belt cover 510 within the set cover area A1.

[0079] In addition, the blocked area may be an area of ​​the pattern P where the pattern P is blocked by the gesture operation.

[0080] In addition, the signal output unit 130 may output an output signal designated for the masked pattern P.

[0081] Specifically, the target region detection unit 100 may be an object detector that individually detects objects shown in an image through an AI-based image recognition algorithm.

[0082] Therefore, if Fig.10 As shown, the target area detection unit 100 detects the mask area A1 including the belt mask 510 in the form of a square frame from the image captured by the image sensor 400 .

[0083] Then, if Fig.11 As shown, the pattern P printed on the tape cover 510 can be detected in the form of a square frame within the detected cover area A1.

[0084] Then, if Fig.12 As shown, when the pattern P among the patterns P printed on the belt cover 510 is hidden by the hand of the passenger, the hidden region detection unit 120 may detect the hidden pattern P.

[0085] Here, an output signal may be specified for each pattern P.

[0086] Therefore, the signal output unit 130 outputs an output signal designated for the mask pattern P, and thus a function of the electronic device corresponding to the output signal may be performed.

[0087] For reference, if two patterns P are blocked together, a function different from that when one pattern P is blocked is set to be executed, thereby increasing the number of executable functions.

[0088] refer to Figure 1 and Figures 13 to 15 The configuration of the second embodiment of the gesture recognition system is described in detail. The target area detection unit 100 can detect the upper body of the passenger based on the captured image of the passenger, set the upper body area A2 as the detection target area, and detect the pattern set on the belt 500 in the set upper body area A2.

[0089] In addition, the blocked area may be an area of ​​the pattern P where the pattern P is blocked by the gesture operation.

[0090] In addition, the signal output unit 130 may output an output signal designated for the mask pattern P.

[0091] Specifically, the target area detection unit 100 may be a passenger detector that detects the body shape and body parts of a passenger through an AI-based image recognition algorithm.

[0092] Here, the passenger detector 310 may use a skeleton-based body keypoint technique to detect the body parts of the passenger, but other algorithms capable of detecting the body parts of the passenger may also be used.

[0093] Therefore, if Fig.13 As shown, from the image captured by the image sensor 400, an upper body area A2 including the upper body of the passenger is detected in the form of a square frame.

[0094] In addition, if Fig.14 As shown, the pattern P printed on the belt cover 510 may be detected in the form of a square frame within the detected upper body region A2.

[0095] Here, although not shown in the drawings, the pattern P may be directly printed on the tape 500 instead of the tape cover 510. In this case, the pattern P printed on the tape 500 may be detected in the form of a square frame.

[0096] Then, if Fig.15 As shown, when the pattern P among the patterns P printed on the belt cover 510 is hidden by the hand of the passenger, the hidden region detection unit 120 may detect the hidden pattern P.

[0097] Here, an output signal may be specified for each pattern P.

[0098] Therefore, the signal output unit 130 outputs an output signal designated for the mask pattern P, and thus a function of the electronic device corresponding to the output signal may be performed.

[0099] refer to Figure 2 and Figures 16 to 18 The configuration of the third embodiment of the gesture recognition system is described in detail. The target area detection unit 100 can detect the belt cover 510 provided on the belt 500 based on the captured image of the passenger, set the cover area A1 including the belt cover 510 as the detection target area, divide the set cover area A1, and detect the divided area.

[0100] Furthermore, a hand detection unit 110 is provided to detect a passenger's hand located within the detection target area according to a gesture operation, and the blocked area may be a segmented area a blocked by the passenger's hand among the segmented areas.

[0101] The signal output unit 130 may output an output signal designated for masking the segmented area a.

[0102] Specifically, the target region detection unit 100 may be an object detector that individually detects objects shown in an image through an AI-based image recognition algorithm.

[0103] In addition, the hand detection unit 110 may be a passenger detector that detects the body shape and body parts of the passenger through an AI-based image recognition algorithm.

[0104] Therefore, if Fig.16 As shown, from the image captured by the image sensor 400, the mask area A1 including the belt mask 510 is detected in the form of a square frame.

[0105] Then, if Fig.17 As shown, the detected mask area A1 is divided into a set number to detect divided areas a.

[0106] Then, if Fig.18 As shown, when the passenger's hand is detected in the cover area A1 and the segmented area a where the belt cover 510 is located is hidden by the passenger's hand, the hidden area detection unit 120 may detect the hidden segmented area a.

[0107] Here, an output signal may be specified for each segmented area a.

[0108] Therefore, the signal output unit 130 outputs an output signal designated for the masked segmented area a, and thus a function of the electronic device corresponding to the output signal may be performed.

[0109] refer to Figure 3 The configuration of the fourth embodiment of the passenger gesture recognition system according to the present disclosure includes: a target area detection unit 200, which detects a detection target area around a belt 500 fastened to a passenger's seat belt; a hand detection unit 210, which detects a hand of the passenger located within the detection target area according to a gesture operation when the passenger makes a gesture; and a signal output unit 220, which outputs an output signal specified for the position of the hand and controls execution of a function regarding the output signal.

[0110] For example, the target area detection unit 200 detects the detection target area based on an image captured by the image sensor 400, and the image sensor 400 may be a visual sensor that detects IR and RGB.

[0111] Since the image sensor 400 has been described above, a redundant description thereof will be omitted.

[0112] In addition, the target area detection unit 200 detects the detection target area limited to the area around the seat belt belt 500. The belt 500 may be a shoulder belt worn on the shoulder and upper body of the passenger.

[0113] Therefore, hardware resources are reduced by reducing the area in which the target area detection unit 200 operates to detect the detection target area, and misdetection and malfunction of the detection target area are prevented by preventing operation in an area without the tape 500 .

[0114] The hand detection unit 210 detects a passenger's hand moving according to a gesture of the passenger (user) based on an image captured by the image sensor 400 .

[0115] The signal output unit 220 is used to output an output signal of the masked area.

[0116] refer to Figure 3 and Figures 19 to 21 The configuration of the fourth embodiment of the gesture recognition system is described in detail. The target area detection unit 200 can detect the upper body of the passenger and set the upper body area A2 as the detection target area, and the hand detection unit 210 can detect the position of the hand on the path of the belt 500 located in the upper body area A2.

[0117] The signal output unit 220 may secure a classification value of the hand position by comparing the coordinates of the detected hand position with designated coordinates of the upper body area A2, and output an output signal designated for the secured classification value.

[0118] Specifically, the target area detection unit 200 may be a passenger detector that detects the body shape and body parts of the passenger through an AI-based image recognition algorithm.

[0119] In addition, the hand detection unit 210 may be a passenger detector that detects the body shape and body parts of the passenger through an AI-based image recognition algorithm.

[0120] Therefore, if Fig.19 As shown, from the image captured by the image sensor 400, the upper body area A2 including the upper body of the passenger is detected in the form of a square frame, and the passenger's hands are also detected in the form of a square frame.

[0121] Then, if Fig. 20 and 21 As shown, when the passenger's hand is detected in the upper body area A2, it is checked whether the passenger's hand is located on the path of the belt 500.

[0122] If the passenger's hand is located on the path of the belt 500 , the signal output unit 220 outputs an output signal designated for the portion where the passenger's hand is located, and thus performs a function of the electronic device corresponding to the output signal.

[0123] refer to Figure 4 and Figure 5 , the configuration of the fifth and sixth embodiments of the passenger gesture recognition system according to the present disclosure includes: a gesture tracking unit 320, which tracks the passenger's gesture moving along the belt 500 fastened to the passenger's seat belt; and a signal output unit 330, which outputs an output signal specified for the gesture movement and controls the execution of a function regarding the output signal.

[0124] That is, since the gesture movement of the passenger is performed along the path of the belt 500, the operation area for gesture tracking is reduced, and thus the hardware resources are reduced. In addition, by issuing a function operation command through a gesture moving along the belt 500, a gesture can be made without being aware of the designated area where the gesture needs to be made, which not only improves the convenience of making a gesture, but also improves the gesture recognition rate.

[0125] refer to Figure 4 and Figures 22 to 24 The configuration of the fifth embodiment of the gesture recognition system is described in detail. The configuration also includes: a target area detection unit 300 that detects a detection target area around the belt 500; and an object detection unit 310 that detects a gesture object located in the detection target area according to a gesture operation, and a gesture tracking unit 320 is configured to track the gesture object moving in the detection target area according to the gesture operation of the passenger.

[0126] For example, the target area detection unit 300 may be a visual sensor that detects the detection target area based on an image captured by the image sensor 400 .

[0127] Furthermore, the target area detection unit 300 detects a detection target area limited to the webbing 500 of the seat belt.

[0128] Therefore, hardware resources are reduced by reducing the area in which the target area detection unit 300 operates to detect the detection target area, and misdetection and malfunction of the detection target area are prevented by preventing operation in an area without the tape 500 .

[0129] The object detection unit 310 may detect a passenger's hand moving within the detection target area according to a gesture of the passenger (user), or detect an object held by the passenger's hand based on an image captured by the image sensor 400 .

[0130] In addition, the target area detection unit 300 can detect the seat belt belt 500 based on the captured image of the passenger and set the belt as the detection target area, and the object detection unit 310 can detect the passenger's hand moving along the belt 500, one of the belt cover 510 and the belt clip as a gesture object.

[0131] Specifically, the target area detection unit 300 may be an object detector that individually detects objects shown in an image through an AI-based image recognition algorithm.

[0132] In addition, when the gesture object is a passenger's hand, the object detection unit 310 can be a passenger detector that detects the passenger's body shape and body parts through an image recognition algorithm, and when the gesture object is an object such as a belt cover 510 or a belt clip, it can be an object detector that individually detects the object shown in the image through an image recognition algorithm.

[0133] Therefore, if Fig. 22 As shown, from the image captured by the image sensor 400, the belt 500 of the seat belt is detected in the form of a square frame, and when the gesture object is designated as the hand of the passenger, the hand of the passenger is detected.

[0134] Additionally, when a passenger's hand is detected on the path of strap 500, the passenger's hand is tracked.

[0135] Therefore, if Fig.23 As shown, when the passenger's hand moves downward on the path of the belt 500 as if sweeping the belt 500, the signal output unit 330 outputs an output designated for the downward movement of the passenger's hand, and thus the function of the electronic device corresponding to the output signal can be executed.

[0136] In addition, if Fig.24 As shown, when the passenger's hand moves upward on the path of the belt 500 as if sweeping the belt 500, the signal output unit 330 outputs an output designated for the upward movement of the passenger's hand, and thus the function of the electronic device corresponding to the output signal can be executed.

[0137] although Fig.23 and 24 The passenger's hand is described as the gesture object, but even when an object such as belt cover 510 or a belt clip is used as the gesture object, the gesture can be recognized in the same manner as described above to operate the device.

[0138] Additionally, in the present disclosure, reflective paint may be applied to the surface of the gesture object.

[0139] That is, when an object such as belt cover 510 or a belt clip is used as a gesture object, the surface of belt cover 510 or the belt clip is coated with a reflective paint, making it easier to detect the gesture object, thereby improving the detection rate.

[0140] Furthermore, in the present disclosure, the signal output unit 330 may determine whether there is an output signal beam designated for the detected hand shape.

[0141] That is to say, Figure 25 to Figure 27 As shown, various hand shapes of the passenger can be detected, and the same gesture, i.e., downward or upward movement along the strap 500, can be made with different hand shapes.

[0142] However, even for the same hand gesture, the number of functions that can be executed can be increased by specifying output signals so that different functions are performed for each hand shape.

[0143] On the other hand, reference Figure 5 The configuration of the sixth embodiment of the gesture recognition system also includes a gesture sensor 300', which is configured to move along the belt 500 by the passenger's gesture movement, and the gesture tracking unit 320 is configured to track the gesture sensor 300' moving along the belt 500 when the passenger makes a gesture.

[0144] For example, the gesture sensor 300 ′ may be a sensor such as a gyro sensor provided inside the belt cover 510 or the belt clip, and the belt cover 510 and the belt clip may be movable along the belt 500 .

[0145] Therefore, when belt cover 510 or belt clip moves along strap 500 by a gesture of a passenger, gesture tracking unit 320 detects movement of a gyro sensor provided inside belt cover 510 or belt clip to recognize the gesture of the passenger.

[0146] Therefore, when the passenger's hand moves downward or upward along the path of the belt 500, the signal output unit 330 outputs an output signal designated for the downward or upward movement of the passenger's hand, and thus a function of the electronic device corresponding to the output signal may be executed.

[0147] In addition, the signal output unit 330 may determine whether there is an output signal beam designated for the movement pattern of the gesture sensor 300 ′.

[0148] That is, various movement patterns of the gesture sensor 300' may be detected, for example, the movement speed of the gesture sensor 300', the direction in which the front of the gesture sensor 300' is facing, etc., and the same gesture, i.e., moving the gesture sensor 300' downward or upward, may be performed while changing the movement pattern of the gesture sensor 300'.

[0149] However, even for the same gesture, the number of functions that can be executed can be increased by specifying an output signal so that a different function is performed for each movement pattern of the gesture sensor 300 ′.

[0150] Hereinafter, an example of a process of recognizing a gesture using the gesture recognition system of the present disclosure will be described.

[0151] <First Embodiment>

[0152] refer to Fig.28 , an object detection algorithm for detecting the seat belt cover 510 and the pattern P is executed in a state where the passenger is wearing the seat belt ( S100 and S101 ).

[0153] According to the operation of the object detection algorithm, it is determined whether the tape cover 510 and the pattern P printed on the tape cover 510 are detected (S102).

[0154] As a result of the determination in step S102 , if the tape cover 510 and the pattern P are detected, the cover area A1 including the tape cover 510 is set as a detection target area ( S103 ).

[0155] Subsequently, when the passenger makes a gesture, it is checked whether there is a pattern P hidden by the gesture among the patterns P printed on the belt cover 510 ( S104 ).

[0156] As a result of the check in step S104, if it is confirmed that there is a blocked pattern P, it is determined whether there is an output signal designated for the blocked pattern P (S105 and S106).

[0157] As a result of the determination in step S106, if there is an output signal for the masked pattern P, the output signal is output (S107).

[0158] Therefore, a function of the electronic device corresponding to the output signal may be performed (S108).

[0159] <Second Embodiment>

[0160] refer to Fig.29 , running a passenger detection algorithm (S200 and S201) for detecting body parts of a passenger when the passenger fastens his seat belt.

[0161] According to the operation of the passenger detection algorithm, the upper body of the passenger is detected, and the upper body region A2 is set as a detection target region (S202).

[0162] Then, an object detection algorithm for detecting the pattern P printed on the seat belt tape 500 is run (S203).

[0163] According to the operation of the object detection algorithm, it is determined whether the pattern P printed on the tape 500 is detected (S204).

[0164] As a result of the determination in step S204, when the pattern P printed on the belt 500 is detected, when the passenger makes a gesture in this state, it is checked whether there is a pattern hidden by the gesture among the patterns P printed on the belt cover 510 (S205).

[0165] As a result of the check in step S205, if it is confirmed that the mask pattern P exists, it is determined whether there is an output signal designated for the mask pattern P (S206 and S207).

[0166] As a result of the determination in step S207, if there is an output signal for the masked pattern P, the output signal is output (S208).

[0167] Therefore, a function of the electronic device corresponding to the output signal may be performed (S209).

[0168] <Third Embodiment>

[0169] refer to Fig.30 , an object detection algorithm for detecting the belt cover 510 of the seat belt when the passenger fastens the seat belt is executed (S300 and S301).

[0170] According to the operation of the object detection algorithm, it is determined whether the belt cover 510 is detected (S302).

[0171] As a result of the determination in step S302, when the belt cover 510 is detected, the cover area A1 including the belt cover 510 is set as a detection target area (S303).

[0172] Then, the mask area A1 is divided by a set number (S304).

[0173] Subsequently, a hand detection algorithm is run to detect the hand position and hand shape of the passenger (S305).

[0174] According to the operation of the hand detection algorithm, it is determined whether the passenger's hand is detected in the cover area A1 (S306).

[0175] As a result of the determination in step S306, when the passenger's hand is detected in the cover area A1, it is determined whether there is a segment area a hidden by the hand in the segment area (a) (S307).

[0176] As a result of the determination in step S307, if it is confirmed that the blocked segmented area a exists on the tape 500, it is determined whether there is an output signal designated for the blocked segmented area a (S308 and S309).

[0177] As a result of the determination in step S309, if there is an output signal designated for the masked segmented area a, the output signal is output (S310).

[0178] Therefore, a function of the electronic device corresponding to the output signal may be performed (S311).

[0179] <Fourth Embodiment>

[0180] refer to Fig.31 , running a passenger detection algorithm for detecting a body part of a passenger when the passenger fastens his seat belt (S400 and S401).

[0181] According to the operation of the passenger detection algorithm, the upper body of the passenger is detected, and the upper body area A2 is set as the detection target area (S402).

[0182] Subsequently, a hand detection algorithm is run to detect the position of the passenger's hand (S403).

[0183] According to the operation of the hand detection algorithm, it is determined whether the passenger's hand is detected on the path of the belt 500 (S404).

[0184] As a result of the determination in step S404, when the passenger's hand is detected, the position of the detected passenger's hand is classified by comparing the coordinates of the position of the detected passenger's hand with the set coordinates of the upper body area A2, and the classification value is checked (S405 and S406).

[0185] Subsequently, it is determined whether there is an output signal designated for a classification value (S407).

[0186] As a result of the determination in step S407, if there is an output signal designated for the classification value, the output signal is output (S408).

[0187] Therefore, a function of the electronic device corresponding to the output signal may be performed (S409).

[0188] <Fifth Embodiment>

[0189] refer to Fig.32 , running an object detection algorithm for detecting seat belts when passengers fasten their seat belts (S500 and S501).

[0190] When the seat belt belt 500 is detected according to the operation of the object detection algorithm, the belt 500 is set as a detection target area (S502).

[0191] Subsequently, a hand detection algorithm is run to detect the passenger's hands (S503).

[0192] According to the operation of the hand detection algorithm, it is determined whether the passenger's hand is detected on the path of the belt 500 (S504).

[0193] As a result of the determination in step S504, when the hand of the passenger is detected, it is determined whether there is an output signal beam designated for the detected shape of the hand of the passenger (S505).

[0194] As a result of the determination in step S505, if it is determined that there is an output signal beam designated for the detected hand shape, the movement of the hand is tracked (S506).

[0195] By tracking the movement of the hand, it is determined whether the hand moves upward or downward along the band 500 (S507).

[0196] As a result of the determination in step S507, if the hand moves upward or downward along strap 500, an output signal designated for the corresponding movement is output (S508).

[0197] Therefore, a function of the electronic device corresponding to the output signal may be performed (S509).

[0198] <Sixth Embodiment>

[0199] refer to Fig.33 , when the passenger fastens the seat belt, a signal from the gesture sensor 300 ′ is detected, and it is determined whether the signal from the gesture sensor 300 ′ is equal to or greater than a threshold value ( S600 and S601 ).

[0200] As a result of the determination of step S601, if the signal from the gesture sensor 300' is equal to or greater than the threshold value, the movement of the gesture sensor 300' is tracked (S602).

[0201] Subsequently, it is determined whether there is an output signal beam designated for the motion pattern of the gesture sensor 300 ′ ( S603 ).

[0202] As a result of the determination in step S603, if it is determined that there is an output signal beam designated for the motion pattern of the gesture sensor 300', an output signal designated for the corresponding motion is output (S604).

[0203] Therefore, a function of the electronic device corresponding to the output signal may be performed (S605).

[0204] The present disclosure specifies that the detection object is limited to the webbing of a seat belt that a passenger always wears.

[0205] Therefore, the present disclosure has the advantages of reducing development costs by simplifying algorithms for gesture recognition and minimizing the capacity of labeled images for DB, and minimizing hardware construction costs by constructing algorithms that can operate sufficiently even with low-quality images.

[0206] Furthermore, since a command is given with a simple gesture covering the tape, it is possible to make a gesture without having to be aware of a specific area where a gesture needs to be made, and thus the convenience of making a gesture and the gesture recognition rate can be improved.

[0207] Although the present disclosure has been described in detail only with respect to the specific examples mentioned above, it is obvious to those skilled in the art that various variations and modifications are possible within the technical scope of the present disclosure, and these variations and modifications naturally fall within the scope of the appended patent claims.

Claims

1. A system for recognizing a gesture of a passenger in a vehicle, the system comprising: The controller includes a target area detection unit configured to: detect a detection target area based on an image captured by the image sensor, the detection target area being located around a belt of a safety belt fastened to a passenger; The obscured area detection unit of the controller is configured to: detect an area within the detection target area that is obscured by the passenger's gesture; and The signal output unit of the controller is configured to output an output signal designated for the masked area and control execution of a function related to the output signal.

2. The system according to claim 1, wherein: The target area detection unit is further configured to: Detecting a pattern in the detection target area.

3. The system according to claim 2, wherein: The pattern is printed on a belt cover provided on the belt, or is printed on the belt.

4. The system according to claim 2, wherein: The target area detection unit is further configured to: detecting a belt cover disposed on the belt based on the captured image of the passenger; setting a mask area including the mask as a detection target area; as well as The pattern provided on the tape cover is detected in the set cover area.

5. The system according to claim 2, wherein: The target area detection unit is further configured to: detecting an upper body of the passenger based on the captured image of the passenger; Set the upper body area as the detection target area; as well as In the set upper body area, the pattern arranged on the belt is detected.

6. The system according to claim 2, wherein: The covered area is an area of ​​the pattern where the pattern covered by the gesture is located.

7. The system according to claim 6, wherein: The signal output unit is configured to: An output signal designated for the mask pattern is output.

8. The system according to claim 1, wherein: The target area detection unit is further configured to: detecting a belt cover disposed on the belt based on the captured image of the passenger; setting a mask area including the mask as a detection target area; dividing the set mask area; and Detect segmented regions.

9. The system according to claim 8, further comprising a hand detection unit, wherein the hand detection unit is configured to: detect a hand of a passenger located within the detection target area through a gesture, in, The covered area is a segmented area covered by the passenger's hand.

10. A vehicle comprising the system according to claim 1.

11. A system for recognizing a gesture of a passenger in a vehicle, the system comprising: The controller includes a target area detection unit configured to: detect a detection target area based on an image captured by the image sensor, the detection target area being located around a belt of a safety belt fastened to a passenger; The hand detection unit of the controller is configured to: when the passenger makes a gesture, detect the hand of the passenger located within the detection target area through the gesture; and The signal output unit of the controller is configured to output an output signal designated for the position of the hand and control execution of a function related to the output signal.

12. The system of claim 11, wherein: The target area detection unit is further configured to: detect the upper body of the passenger and set the upper body area as the detection target area, and The hand detection unit is further configured to detect a position of a hand on a path of the strap within the upper body region.

13. The system according to claim 12, wherein: The signal output unit is further configured to: Obtaining a classification value of the hand position by comparing the detected coordinates of the hand position with the designated coordinates of the upper body region; and Output is the output signal specified for the resulting classification value.

14. A system for recognizing a gesture of a passenger in a vehicle, the system comprising: a controller including a gesture tracking unit configured to: track the movement of the gesture within a detection target area based on an image captured by the image sensor when the passenger performs a gesture moving along a belt of a seat belt fastened to the passenger; and The signal output unit of the controller is configured to output an output signal designated for the gesture movement and control execution of a function related to the output signal.

15. The system of claim 14, further comprising: A target area detection unit is configured to: detect a detection target area around the belt; and an object detection unit, configured to: detect a gesture object located in the detection target area through a gesture, Wherein, the gesture tracking unit is further configured to: track a gesture object moving within the detection target area through a passenger's gesture.

16. The system of claim 15, wherein: The target area detection unit is further configured to: detect a belt of the seat belt based on the captured image of the passenger and set the belt as a detection target area, and The object detection unit is further configured to detect one of a hand of a passenger moving along the belt, a belt cover, and a belt clip as a gesture object.

17. The system of claim 16, wherein: The surface of the gesture object is coated with reflective paint.

18. The system of claim 16, wherein: The signal output unit is further configured to determine whether there is an output signal beam designated for the detected hand shape.

19. The system of claim 14, further comprising a gesture sensor configured to be movable along the belt by a gesture of a passenger, in, The gesture tracking unit is further configured to track the gesture sensor moving along the strap when a passenger makes a gesture.

20. The system of claim 19, wherein: The signal output unit is further configured to determine whether there is an output signal beam designated for the movement pattern of the gesture sensor.

Images