Seat belt wearing determination device

Abstract

The present disclosure provides a seat belt wearing determination device including a camera configured to capture an interior of a vehicle, and a controller configured to determine a first bounding box and a second bounding box from an image including a seated occupant captured by the camera, and determine whether the seated occupant wears a seat belt based on an overlapping degree of the first bounding box and the second bounding box, wherein the first bounding box corresponds to an upper body region of the seated occupant, and the second bounding box includes the seat belt detected from the captured image by the controller.

Description

Technical Field

[0001] This disclosure relates to methods and devices for determining whether a seated occupant is wearing a seatbelt based on video information analysis. Background Technology

[0002] Today, most vehicles are equipped with seat belts to ensure the safety of occupants in the event of a vehicle collision. Seat belts (often also referred to as seat belts) play a crucial role in minimizing occupant injuries caused by a collision by preventing occupants from being rapidly separated from their seats and from impacting the vehicle's internal structure or ejecting from the vehicle.

[0003] These seat belts restrain occupants' bodies to prevent them from being pulled out of their seats, but this can create pressure and discomfort when occupants feel the restraint caused by the seat belts. As a result, despite the presence of seat belts, there is a possibility of serious injury if an occupant chooses not to wear one.

[0004] To address this issue, various technologies have been developed to encourage occupants to wear seat belts. One example of such technology is using weight information to incentivize seat belt use. This technology determines the presence of an occupant by detecting the weight applied to the seating surface via a switch sensor mounted on the seat cushion, and assesses the seat belt's tightness via a buckle sensor mounted on the seat belt. This allows for the identification of situations where an occupant is present but the seat belt is not fastened, prompting the occupant to fasten their seat belt by sending an alarm.

[0005] However, when occupants intentionally use additional tools such as dummy buckles to deceive the sensors, or when the seatbelt is worn incorrectly, this technology may mistakenly identify the seatbelt as fastened, failing to properly restrain the body, contrary to its intended use. Therefore, there is a possibility that the seatbelt-encouraging function may not work effectively, as the system can determine that the seatbelt is fastened even when the occupant is not actually wearing it.

[0006] The above description of the background art is intended to enhance the understanding of the background of this disclosure and should not be construed as an admission that it is prior art known to those skilled in the art. Summary of the Invention

[0007] This disclosure relates to a method and apparatus for determining whether a seated person is wearing a seatbelt based on the analysis of video information.

[0008] The technical objectives intended to be achieved in this disclosure are not limited to those described above, and other technical objectives not described herein will be clearly understood by those skilled in the art from the following description.

[0009] To achieve the above objectives, a seatbelt wearing determination device according to an embodiment of the present disclosure includes: a camera configured to capture images of the interior of a vehicle; and a controller configured to determine a first bounding box and a second bounding box from images captured by the camera including a seated occupant, and to determine whether the seated occupant is wearing a seatbelt based on the degree of overlap between the first bounding box and the second bounding box, wherein the first bounding box corresponds to the upper body area of ​​the seated occupant, and the second bounding box includes the seatbelt detected by the controller from the captured images.

[0010] For example, the controller can detect at least one predetermined point in the upper body area of ​​the seated occupant from the captured image and determine the first bounding box based on the detected point.

[0011] For example, the predetermined point may include a point corresponding to the center of the upper body of the seated occupant's shoulders and abdomen.

[0012] For example, the controller may determine the width of the first bounding box based on points corresponding to the shoulders, and determine the height of the first bounding box based on at least one of the points corresponding to the shoulders and the point corresponding to the center of the abdomen.

[0013] For example, the controller may, in response to the absence of a portion of the at least one predetermined point, derive the undetected point based on other detected points, and determine the first bounding box by further considering the derived points.

[0014] For example, the controller can extract seat belt feature information from the captured image, obtain classification information from the extracted seat belt feature information, and detect the seat belt based on the obtained classification information.

[0015] For example, the controller can determine whether the seatbelt is being worn by comparing the degree of overlap with a predetermined threshold.

[0016] For example, the controller may determine that the seated occupant is not wearing the seat belt in response to the overlap being equal to or less than the predetermined threshold.

[0017] For example, the controller can determine the category of the seated occupant and determine the first bounding box based on the determined category of the seated occupant.

[0018] For example, the controller can obtain the seat position of the seat, determine the area value of the seat based on parameters associated with the seat position, detect the seated occupant from the captured image, determine the area value of the seated occupant, and determine the category of the seated occupant based on the area ratio between the area value of the seat and the area value of the seated occupant on the corresponding seat.

[0019] For example, the seat position may include the fore-and-aft position of the seat, and the parameter may be pre-configured to correspond at least to the foremost and rearmost positions of the seat.

[0020] For example, the seat position may include the seat back recline angle, and the controller may additionally consider the recline angle to determine the seat area value.

[0021] For example, the controller may determine the category as a predetermined category in response to the absence of the seated occupant in the captured image.

[0022] For example, the controller can obtain a seated occupant segment from the captured image and determine the area value of the seated occupant based on the obtained seated occupant segment.

[0023] For example, the controller can compare the area ratio with a predetermined threshold and determine the category of the seated occupant based on the comparison result.

[0024] For example, the controller may determine the category of the seated occupant based on multiple categories defined by a range between two adjacent thresholds from a plurality of predetermined thresholds.

[0025] For example, the controller can obtain the weight information of the seated occupant and determine the category of the seated occupant based on additional considerations of the weight information.

[0026] For example, the controller can obtain the weight information of the seated occupant from a weight sensor connected to the seat.

[0027] For example, the controller can determine the area ratio of each of the plurality of seats, and determine the category of the seated occupant of each of the plurality of seats based on the area ratio determined for each seat.

[0028] For example, the controller can determine at least one or a combination of the first bounding box and the second bounding box by inputting the captured image into an artificial intelligence algorithm, wherein the artificial intelligence algorithm includes at least one of machine learning, neural networks, deep learning, classification algorithms, and combinations thereof.

[0029] The seatbelt wearing determination method described in at least one embodiment of this disclosure can accurately assess whether a seatbelt is being worn correctly for its intended purpose by analyzing video information.

[0030] Furthermore, by utilizing bounding boxes corresponding to seated occupants and seat belts on the video information, the accuracy and speed of video information analysis used for seat belt wearing determination can be improved.

[0031] By utilizing video information, seated passengers can be detected and classified more accurately.

[0032] By improving the accuracy of occupant detection, problems can be mitigated where functions installed in vehicles may malfunction or fail to operate as expected due to incorrect detection.

[0033] By improving the classification of seated occupants, the functions installed in the vehicle can be customized to accurately reflect the specific characteristics of each seated occupant, thereby improving performance.

[0034] The seatbelt wearing determination method described in at least one embodiment of this disclosure is able to accurately assess whether a seatbelt is being worn correctly for its intended purpose by analyzing video information.

[0035] Furthermore, by utilizing bounding boxes corresponding to seated occupants and seat belts on the video information, the accuracy and speed of video information analysis used for seat belt wearing determination can be improved.

[0036] By utilizing video information, seated passengers can be detected and classified more accurately.

[0037] By improving the accuracy of occupant detection, problems can be mitigated where functions installed in vehicles may malfunction or fail to operate as expected due to incorrect detection.

[0038] By improving the classification of seated occupants, the functions installed in the vehicle can be customized to accurately reflect the specific characteristics of each seated occupant, thereby improving performance.

[0039] The advantages of this disclosure are not limited to those described above, and other advantages not described herein will be readily apparent to those skilled in the art from the following description. Attached Figure Description

[0040] Figure 1 This is a block diagram illustrating the configuration of a seatbelt wearing determination device according to an embodiment of the present disclosure;

[0041] Figure 2 This is a diagram illustrating the process of deriving the overlap ratio between bounding boxes for determining seat belt wearing according to embodiments of the present disclosure;

[0042] Figure 3 This is a diagram illustrating a method for determining seatbelt wearing based on overlap ratio according to an embodiment of the present disclosure;

[0043] Figure 4 This is a flowchart of a seatbelt wearing determination method according to an embodiment of the present disclosure;

[0044] Figure 5 This is a block diagram illustrating the configuration of a seatbelt wearing determination device according to another embodiment of the present disclosure;

[0045] Figure 6 This is a diagram showing the interior of a vehicle equipped with seat belts according to an embodiment of the present disclosure;

[0046] Figure 7 This is a diagram illustrating the determination of seated occupant area values ​​for classifying seated occupants according to an embodiment of the present disclosure;

[0047] Figure 8 This is a diagram illustrating the classification of seated occupants based on area ratio and threshold according to embodiments of the present disclosure; and

[0048] Figure 9 This is a flowchart illustrating the process of classifying seated occupants in a seatbelt wearing determination method according to an embodiment of the present disclosure. Detailed Implementation

[0049] The specific structural or functional descriptions of the embodiments of this disclosure disclosed in this specification or patent application are illustrative examples intended to describe the embodiments of this disclosure, and the embodiments of this disclosure may be implemented in various forms and should not be construed as limited to those described in this specification or application.

[0050] Various modifications and different forms can be made to the embodiments of this disclosure, therefore specific embodiments are shown in the accompanying drawings and described in detail in this specification or application. However, this should not be construed as limiting the embodiments of this disclosure to the specific forms disclosed, but should be understood to include all modifications, equivalents, or substitutions falling within the conceptual and technical scope of this disclosure.

[0051] Unless otherwise defined, all terms used herein (including technical or scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms (including those defined in common dictionaries) should be interpreted in a manner consistent with their meaning in the context of the relevant field and should not be interpreted in an idealized or overly formal sense, unless expressly defined in this specification.

[0052] Hereinafter, embodiments disclosed in this specification will be described with reference to the accompanying drawings, in which the same reference numerals denote the same or similar parts, and redundant descriptions are omitted.

[0053] As used in the following description, the suffixes “module” and “unit” are granted or used interchangeably for ease of description, but they do not have a specific meaning or function in themselves.

[0054] Furthermore, detailed descriptions of well-known technologies related to the embodiments disclosed in this specification may be omitted to avoid obscuring the subject matter of the embodiments disclosed in this specification. Additionally, the accompanying drawings are provided only for easy understanding of the embodiments disclosed in this specification and do not limit the technical spirit disclosed herein. It should be understood that the embodiments include all modifications, equivalents, and substitutions within the spirit and scope of this disclosure.

[0055] As used herein, terms including ordinal numbers such as “first” and “second” can be used to describe various components without limiting them. These terms are only used to distinguish one component from another.

[0056] It should be understood that when a component is described as "connected to" or "linked to" another component, it can be directly connected to or linked to the other component, or there may be an intermediate component. Conversely, when a component is described as "directly connected to" or "directly linked to" another component, there is no intermediate component.

[0057] As used in this article, the singular form is also intended to include the plural form, unless the context clearly indicates otherwise.

[0058] It will also be understood that, when used in this specification, the terms “comprising” or “having” specify the presence of the said features, numbers, steps, operations, components, elements or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, elements or combinations thereof.

[0059] For example, each controller may include a communication device that communicates with another controller or sensor to control the functions it is responsible for, a memory that stores operating system or logic instructions and input / output information, and one or more processors for determining, operating, and making decisions required for the functions it is responsible for.

[0060] Embodiments of this disclosure provide a seatbelt wearing determination method and apparatus based on video information analysis, which can more accurately determine whether a seatbelt is being worn and assess whether the seatbelt is being worn correctly according to its intended purpose based on a bounding box corresponding to the seated occupant and the seatbelt.

[0061] Before explaining the method for determining seatbelt wearing according to this disclosure, reference will be made to... Figure 1A seatbelt wearing determination device according to an embodiment of the present disclosure is described.

[0062] Figure 1 This is a diagram illustrating the configuration of a seatbelt wearing determination device according to an embodiment of the present disclosure.

[0063] Reference Figure 1 According to one embodiment, the seatbelt wearing determination device 100 may include a camera 140 and a controller 150. Figure 1 The main components shown are those relevant to embodiments of this disclosure, and it will be apparent to those skilled in the art that a practical implementation of device 100 may include additional components beyond those depicted in the figures. Each component will be described below.

[0064] First, camera 140 is used to capture images of the vehicle's interior and transmit the captured images to controller 150 and other devices. To facilitate determination of seatbelt 10 wearing, camera 140 is preferably positioned at the front of the vehicle, facing rearward, to ensure effective capture of the seated occupant's forward view. In this case, camera 140, which captures images of the vehicle's interior, can be implemented as an in-cabin camera. Simultaneously, when implemented as camera 140, the in-cabin camera is installed inside the vehicle and allows for real-time monitoring of the seated occupant's condition, facilitating continuous monitoring of whether the seatbelt 10 is being worn by utilizing continuously provided images.

[0065] Simultaneously, the controller 150 acquires images containing the seated occupant from the camera 140 and determines whether the seated occupant is wearing the seatbelt 10 based on these images. The controller 150 determines a first bounding box 210 and a second bounding box 220 from the acquired images, and determines whether the seated occupant is wearing the seatbelt 10 based on the overlap ratio between the determined bounding boxes. In this case, the first bounding box 210 corresponds to the upper body area of ​​the seated occupant in the acquired image, while the second bounding box 220 surrounds the seatbelt 10 detected from the acquired image. (Reference) Figure 2 and Figure 3 A detailed description is provided of how the controller 150 determines whether a seated occupant is wearing a seatbelt 10.

[0066] Figure 2 This is a diagram illustrating the process of deriving the overlap ratio between bounding boxes for determining seat belt wearing according to embodiments of the present disclosure.

[0067] Reference Figure 2 The image obtained from camera 140 shows the seated occupant, seat belt 10, first bounding box 210, second bounding box 220, and overlapping area 230.

[0068] First, the process of determining a first bounding box 210 corresponding to the upper body region of the seated occupant from the acquired image will be described. The controller 150 can detect at least one preset point from the acquired image to determine the first bounding box 210, which is included in the upper body region of the seated occupant. In this case, to ensure that the first bounding box 210 captures the upper body region of the seated occupant, each point can be preset to correspond to a predetermined position on the upper body of the seated occupant in the image, such as the left shoulder 211, right shoulder 212, and abdominal center 213.

[0069] The controller 150 can determine the first bounding box 210 based on the detected points. Specifically, when points corresponding to the shoulders 211 and 212 and the center of the abdomen 213 of a seated occupant are detected in the acquired image, the width of the first bounding box 210 can be determined based on the points corresponding to the left shoulder 211 and the right shoulder 212. Furthermore, the height of the first bounding box 210 can be determined based on at least one of the points corresponding to the shoulders 211 and 212 and the point corresponding to the center of the abdomen 213. For example, the controller 150 can determine the width of the first bounding box 210 as the horizontal distance between the points corresponding to the shoulders 211 and 212 and the center of the abdomen 213, and determine the height of the first bounding box 210 as the vertical distance between the points corresponding to the shoulders 211 and 212 and the center of the abdomen 213. By determining the first bounding box 210 in this way, it is possible to prevent the first bounding box 210 from being determined to be too small or too large compared to the upper body area of ​​the seated occupant in the image, thereby ensuring that the first bounding box appropriately corresponds to the upper body area of ​​the seated occupant.

[0070] Simultaneously, when the upper body of a seated occupant is partially obscured due to the occupant's posture or other obstacles, some predetermined points may not be detectable from the acquired image. In this case, the controller 150 can derive undetected points based on the remaining detected points and determine the first bounding box 210 by considering both the derived and detected points. For example, when the point corresponding to the seated occupant's left shoulder 211 is obscured and not detected, the controller 150 can derive the point corresponding to the left shoulder 211 by calculating the horizontal distance between the point corresponding to the abdominal center 213 or sternum and the right shoulder 212, and then, assuming the shoulders are symmetrical, horizontally shift the calculated horizontal distance from the point corresponding to the abdominal center 213 or sternum in the left direction. In addition to the above method, undetected points can also be derived by utilizing stored or separately received human body information.

[0071] Simultaneously, the controller 150 can detect the seat belt 10 from the acquired image and determine a second bounding box 220 including the detected seat belt 10. To achieve this, the controller 150 can extract feature information of the seat belt 10 from the acquired image, obtain classification information from the extracted feature information, and detect the seat belt based on the obtained classification information. In this case, the feature information of the seat belt 10 may include attributes such as color, shape, and position, and the obtained classification information can facilitate the differentiation of the seat belt 10 from other objects in the image.

[0072] Simultaneously, the determination of the first bounding box 210 and the second bounding box 220 can be performed by inputting the acquired image into a specified artificial intelligence learning model. Here, the artificial intelligence learning model may include at least one of machine learning, neural networks, deep learning, classification algorithms, and combinations thereof as an artificial intelligence algorithm.

[0073] Specifically, human pose estimation from an artificial intelligence learning model can be used to determine the first bounding box 210. Human pose estimation is the process of detecting multiple key points corresponding to important body parts of a person and using them to estimate the pose of the object. In embodiments of this disclosure, detected key points belonging to the upper body of a seated occupant can be used as points for determining the first bounding box 210. Furthermore, in the detection of the seat belt 10, classification from an artificial intelligence learning model can be used to improve detection accuracy, and bounding box regression from an artificial intelligence learning model can be used to compensate for the first bounding box 210 and the second bounding box 220 to have more accurate values.

[0074] Once the first bounding box 210 and the second bounding box 220 are determined through this process, the controller 150 can evaluate the degree of overlap between the first bounding box 210 and the second bounding box 220. Here, the degree of overlap can be determined based on the area of ​​the region 230 included in both the first bounding box 210 and the second bounding box 220 on the acquired image.

[0075] By analyzing the images obtained through camera 140, such as Figure 2 As shown, even if the buckle sensor indicates that the seat belt is not being worn, it can be determined that the seat belt is not being worn when the seated occupant is not actually wearing the seat belt 10. This facilitates the practical use of the seat belt 10.

[0076] Meanwhile, when image segmentation is used to determine the use of the seat belt 10 based on the region of the seat belt 10 in the acquired image, a large amount of information equal to the number of pixels in the image is required, which may potentially overload the controller 150. On the other hand, according to embodiments of this disclosure, by utilizing the first bounding box 210 and the second bounding box 220, the use of the seat belt 10 can be determined based solely on information from the four corner points of the bounding boxes, as well as width and height information. By doing so, the amount of information required for determination can be reduced and the speed of determination processing can be improved.

[0077] In the following text, reference will be made to Figure 3 The process described above for determining the use of seat belt 10 based on the calculated degree of overlap is described.

[0078] Figure 3 This is a diagram illustrating a method for determining seatbelt wearing based on overlap ratio according to embodiments of the present disclosure.

[0079] Figure 3 A graph is shown with overlap as the vertical axis and time as the horizontal axis to illustrate the method for determining seatbelt wearing according to an embodiment. The graph depicts a schematic representation without precise values ​​to aid understanding.

[0080] First, as can be seen from the graph, the degree of overlap is not a fixed value but varies. Although the degree of overlap is not determined by time, the movement of a seated occupant who is not a stationary object may cause changes in the area where the seated occupant and seat belt 10 appear in the image, resulting in changes in the degree of overlap between the first bounding box 210 and the second bounding box 220.

[0081] In embodiments of this disclosure, determining seatbelt wearing involves camera 140 continuously capturing images of the seated occupant and transmitting the captured images in real time to controller 150, allowing controller 150 to assess whether the seated occupant is wearing seatbelt 10 based on the received images. By doing so, it is possible to continuously monitor whether the seated occupant is wearing seatbelt 10, as shown in the graph.

[0082] Meanwhile, the controller 150 can compare the degree of overlap between the first bounding box 210 and the second bounding box 220 with a predetermined threshold, and determine whether to wear the seat belt 10 based on the comparison result.

[0083] In this configuration, the first bounding box corresponds to the upper body area of ​​the seated occupant, while the second bounding box surrounds the seatbelt 10, resulting in a higher likelihood that the seated occupant is wearing the seatbelt 10 when the degree of overlap increases. Based on this, the controller 150 can determine that the seated occupant is wearing the seatbelt 10 in response to an overlap degree greater than a predetermined threshold, and determine that the seated occupant is not wearing the seatbelt 10 in response to an overlap degree equal to or less than the predetermined threshold.

[0084] The above process is by Figure 4 The flowchart shown is illustrated below. Refer to [the following text] for further details. Figure 4 A method for determining the wearing status of a seatbelt 10 according to an embodiment of the present disclosure is described.

[0085] Figure 4 This is a flowchart of a seatbelt wearing determination method according to an embodiment of the present disclosure.

[0086] Reference Figure 4 In step S401, camera 140 captures an image of the vehicle interior, including the seated occupants, and controller 150 acquires the captured image. Here, to facilitate determining whether the seatbelt 10 is being worn, it is preferable to position camera 140 at the front of the vehicle, facing rearward, to ensure an effective frontal view of the seated occupants. In this case, camera 140, which captures the vehicle interior, can be implemented as an in-cabin camera. When implemented as camera 140, the in-cabin camera is installed inside the vehicle and allows for real-time monitoring of the seated occupants' status, facilitating continuous monitoring of whether the seatbelt 10 is being worn by utilizing the provided images.

[0087] In step S402, the controller 150 that has acquired the image can detect at least one preset point from the upper body area of ​​the seated occupant in the acquired image to determine the first bounding box 210. In this case, in order to ensure that the first bounding box 210 captures the upper body area of ​​the seated occupant, each point can be preset to correspond to a predetermined position on the upper body of the seated occupant in the image, such as the left shoulder 211, the right shoulder 212, and the center of the abdomen 213.

[0088] Next, in step S404, the controller 150 determines a first bounding box 210 based on the detected points. In this case, when the detected points correspond to the shoulders 211 and 212 and the center of the abdomen of the seated occupant, the width of the first bounding box 210 can be determined based on the points corresponding to the left shoulder 211 and the right shoulder 212, and the height of the first bounding box 210 can be determined based on at least one of the points corresponding to the shoulders 211 and 212 and the point corresponding to the center of the abdomen 213. For example, the controller 150 can determine the width of the first bounding box 210 as the horizontal distance between the points corresponding to the shoulders 211 and 212, and the height of the first bounding box 210 as the vertical distance between the points corresponding to the shoulders 211 and 212 and the center of the abdomen 213. By determining the first bounding box 210 in this way by the controller 150, it is possible to prevent the first bounding box 210 from being determined to be too small or too large compared to the upper body area of ​​the seated occupant in the image, thereby ensuring that the first bounding box appropriately corresponds to the upper body area of ​​the seated occupant.

[0089] Meanwhile, when the upper body of a seated occupant is partially obscured due to the occupant's posture or other obstacles, some predetermined points may not be detectable from the acquired image. In this case, the controller 150 can derive the undetected points based on other detected points and determine the first bounding box 210 by considering both the derived and detected points. For example, when the point corresponding to the seated occupant's left shoulder 211 is obscured and not detected, the controller 150 can derive the point corresponding to the left shoulder 211 by calculating the horizontal distance between the point corresponding to the abdominal center 213 or sternum and the right shoulder 212, and then, assuming the shoulders are symmetrical, horizontally shift the calculated horizontal distance from the point corresponding to the abdominal center 213 or sternum in the left direction. In addition to the above method, undetected points can also be derived by utilizing stored or separately received human body information.

[0090] In step S403, the controller 150 further detects the seatbelt 10 from the acquired image to determine a second bounding box 220. The controller 150 can extract feature information of the seatbelt 10 from the acquired image, obtain classification information from the extracted feature information, and detect the seatbelt based on the obtained classification information. In this case, the feature information of the seatbelt 10 may include attributes such as color, shape, and position, and the obtained classification information can facilitate the differentiation of the seatbelt 10 from other objects in the image.

[0091] When the seat belt 10 is detected, in step S405, the controller 150 determines a second bounding box surrounding the detected seat belt 10, and in this case, it is appropriate to determine the second bounding box as a rectangular region having the minimum possible area including the seat belt 10.

[0092] Simultaneously, the aforementioned processing steps S402 to S405 can be performed by inputting the acquired image into a specified artificial intelligence learning model. Here, the artificial intelligence learning model may include at least one of machine learning, neural networks, deep learning, classification algorithms, and combinations thereof as an artificial intelligence algorithm. In particular, the human pose estimation of the artificial intelligence learning model can be used to determine the first bounding box 210. Human pose estimation is the process of detecting multiple key points corresponding to important body parts of a person and using them to estimate the pose of the object, and in embodiments of this disclosure, the detected key points belonging to the upper body of the seated occupant can be used as points for determining the first bounding box 210. Furthermore, in the detection of the seat belt 10, classification by the artificial intelligence learning model can be used to improve detection accuracy, and bounding box regression by the artificial intelligence learning model can be used to compensate for the first bounding box 210 and the second bounding box 220 to have more accurate values.

[0093] Once the first bounding box 210 and the second bounding box 220 are determined, the controller 150 can evaluate the degree of overlap between the two boxes 210 and 220 in step S406, and determine whether the seated occupant is wearing the seat belt 10 based on the degree of overlap. Here, the degree of overlap can be determined based on the area of ​​the region 230 included in the first bounding box 210 and the second bounding box 220 on the acquired image.

[0094] The controller 150 can determine the wearing status of the seat belt 10 by comparing the degree of overlap with a predetermined threshold. Specifically, in step S409, it determines that the seated occupant is wearing the seat belt 10 in response to the degree of overlap being greater than the threshold ("yes" in step S407), and in step S408, it determines that the seated occupant is not wearing the seat belt 10 in response to the degree of overlap being equal to or less than the threshold ("no" in step S407).

[0095] Furthermore, according to embodiments of this disclosure, the process of classifying seated occupants may involve determining the seatbelt wearing status. Specifically, embodiments of this disclosure propose using image information to detect occupants based on the relative area of ​​the seat according to the seat position and occupant area in the image, thereby improving the detection accuracy of seated occupants and enabling more detailed classification of occupant types.

[0096] Before explaining the seated occupant classification method according to the embodiments of this disclosure, reference will first be made to... Figure 5 The configuration of a seatbelt wearing determination device according to an embodiment of this disclosure is described.

[0097] Figure 5 This is a block diagram illustrating the configuration of a seatbelt wearing determination device according to another embodiment of the present disclosure.

[0098] Reference Figure 5 According to the embodiment, the seat belt wearing determination device 100 may include a camera 140, a controller 150, a seat 110, a position sensor 120, and a weight sensor 130. Figure 5 The main components shown are those related to embodiments of this disclosure, and it will be apparent to those skilled in the art that the actual implementation of the seatbelt wearing determination device 100 can be achieved with more or fewer components. The components will be described below.

[0099] First, the seatbelt wearing determination device 100 according to the embodiment includes at least one seat 110, and the seat 110 can be distinguished based on the seating positions, such as the driver's seat 111 and the front passenger seat 112. Meanwhile, in the embodiments of this disclosure, the position of the seat 110 is adjustable. The seat position can include at least one of a position in the driving direction and a direction parallel to the driving direction of the seat 110 arranged in the driver's seat 111, the front passenger seat 112, and the seat back angle. Here, the seat position can refer to the state of the seat 110, which includes the position of the seat 110 within a certain range achieved by moving the seat 110 back and forth via a mechanism such as a seat track. Furthermore, the seat back angle can refer to the angle of the seat back relative to the seating surface or the ground, which varies in the reclined or relaxed mode of the seat 110.

[0100] Furthermore, according to embodiments of this disclosure, the seatbelt wearing determination device 100 may also include a position sensor 120 connected to the seat to detect the seat position. For example, the position sensor 120 may include a seat track position sensor that detects the forward and backward positions of the seat 110 on the seat track. By utilizing such a position sensor 120, the position of the seat 110 can be detected more accurately, thereby facilitating the classification of seated occupants. Moreover, by continuously monitoring changes in the position of the seat 110 via the position sensor 120, the classification of seated occupants can be updated based on these changes.

[0101] Furthermore, according to embodiments of this disclosure, the seatbelt wearing determination device 100 may include a weight sensor 130 connected to the seat 110 to detect the weight applied to the seat 110 and generate weight information of the seated occupant. The weight sensor 130 may be implemented using an occupant classification system (OCS) or an occupant detection system (ODS). When equipped with the weight sensor 130, the seatbelt wearing determination device 100 can utilize weight information in addition to image information to improve the accuracy of occupant classification.

[0102] Simultaneously, camera 140 captures images of the vehicle's interior and transmits the captured images. The images captured by camera 140 may include at least one seat 110 or a seated occupant seated with the seat 110. Furthermore, camera 140 is adapted to be mounted at the front of the vehicle interior facing rearward, thereby allowing a clear representation of the front of the seat 110 and the seated occupant. For example, camera 140 can be implemented as an in-cabin camera. Additionally, the in-cabin camera is capable of real-time monitoring of the occupant's status within the vehicle, allowing the seatbelt wearing determination device 100 according to embodiments of this disclosure to continuously monitor occupant entry and exit, and any changes, using real-time images.

[0103] Images captured by camera 140 may include seat 110 and the seated occupant. When seat 110 undergoes a change in position, such as moving forward or backward or tilting back, while camera 140 remains stationary, the area occupied by seat 110 in the captured image will change significantly. Even if the change in position does not affect the actual area of ​​seat 110 itself, when seat 110 is positioned in front rather than behind, seat 110 will be closer to camera 140, resulting in it occupying a larger area in the captured image. Furthermore, when the seat back has a smaller tilt angle, it will occupy a larger area in the captured image compared to when the seat back has a larger tilt angle. Therefore, considering the position of seat 110 when determining the area of ​​the seat in the captured image yields more accurate results.

[0104] Meanwhile, the controller 150 can obtain the seat position from the position sensor 120 and acquire images of the vehicle interior from the camera 140.

[0105] The controller 150 can determine the seat area value based on parameters associated with the seat position, for example by extracting parameters corresponding to the current position of seat 110 from the acquired seat position and using said parameters to determine the seat area. Here, the area value of seat 110 may represent the area of ​​seat 110 in an image, rather than the actual physical area of ​​seat 110.

[0106] In addition, the controller 150 can detect seated occupants from the acquired interior images of the vehicle and determine the area value corresponding to the detected seated occupants.

[0107] Once the area value of seat 110 and the area value of the occupant seated in that seat are determined, controller 150 can evaluate the area ratio between them and classify the occupant's category or type based on the area ratio. Here, the area ratio can be determined by dividing the occupant's area value by the seat's area value, meaning that a higher ratio corresponds to a relatively large occupant, while a lower ratio indicates a relatively small occupant.

[0108] Due to the influence of the distance and angle between the seat and the camera, changing the position of seat 110 can alter not only the area value of seat 110 but also the area value of the seated occupant within the same occupant's image. Therefore, considering only the area of ​​the seated occupant, the area value of the seated occupant within the image can vary depending on the position of seat 110, making it difficult to classify the seated occupant based on a fixed standard. On the other hand, determining the category of the seated occupant based on the aforementioned area ratio allows for consistent classification standards and improves the accuracy of seated occupant classification.

[0109] The following is for reference Figures 6 to 8The document details the determination of the area values ​​of seat 110 and the seated occupants, as well as the classification of seated occupants based on area ratios.

[0110] Figure 6 This is a diagram showing the interior of a vehicle including a seat according to an embodiment of the present disclosure.

[0111] Reference Figure 6 The image shows the driver's seat 111 and the front passenger seat 112. It can be seen that although the actual dimensions of the driver's seat 111 and the front passenger seat 112 are the same, the driver's seat 111 occupies a larger area in the image compared to the front passenger seat 112. This is because the driver's seat 111 is positioned more forward and closer to the camera 140 compared to the front passenger seat 112. Taking these factors into account, the controller 150 acquires the position of the seat 110 and determines the area value of the seat 110 based on relevant parameters.

[0112] Here, the parameters used to determine the area value of seat 110 based on its position are preferably set to ensure that the area value of seat 110 increases in response to forward positioning and decreases in response to rearward positioning. In this case, these parameters can be configured to correspond at least to the foremost and rearmost positions of seat 110, thereby allowing controller 150 to determine the area value of seat 110 based on its fore-and-aft position.

[0113] The position of the seat can include not only the fore-and-aft position of the seat 110, but also the reclining angle of the seat back. In this case, by taking into account the reclining angle of the seat back in addition to the fore-and-aft position of the seat 110, the controller 150 can more accurately determine the area value of the seat 110 based on the area occupied by the seat 110 in the actual image.

[0114] Already referred to Figure 6 The area value of seat 110 is described relative to the position of seat 110 in this disclosure, and is referred to hereinafter. Figure 7 The description determines the area value for the seated occupants.

[0115] Figure 7 This is a diagram illustrating the determination of the area values ​​of seated occupants for classifying seated occupants according to an embodiment of the present disclosure.

[0116] Figure 7 An interior image of the vehicle, including seat 110 and seated occupants, is displayed. Controller 150 detects seated occupants from the acquired image and determines the area value of the detected seated occupants.

[0117] Here, occupant detection refers to identifying seated occupants appearing in an image, which can be accomplished using an artificial intelligence learning model. Specifically, this model can include deep learning. When using deep learning, seated occupants can be designated as regions of interest (ROIs), and occupant detection can be performed through ROI-based exploration.

[0118] To perform ROI exploration, training data including images and corresponding category and location information is prepared. In embodiments of this disclosure, images of seated occupants and their corresponding categories (e.g., adults, children, etc.) and the positions of the seated occupants in the images can be prepared. During deep learning, the learning model learns the features of the seated occupants based on this information, and iterative learning can be performed to improve accuracy. After the deep learning process, the learned model can receive, for example... Figure 7 The image shown is explored to identify regions that match the features of a seated occupant. When a region matching the features of a seated occupant is found, the learned model can detect that region as a seated occupant.

[0119] When no seated occupant is detected, the controller 150 can infer that there is no occupant in seat 110 and classify the state as, for example, "unoccupied" or "empty". In particular, when the acquired image contains multiple seats 110 and some of these seats 110 do not have occupants, it can be determined which seats 110 have occupants and which seats 110 do not have occupants.

[0120] Simultaneously, the area value of the seated occupant can be determined through seated occupant segmentation. For this purpose, the controller 150 can obtain seated occupant segments from the acquired image and determine the area value of the seated occupant based on them. Here, segmentation refers to the process of classifying objects appearing in an image into different categories, and it includes segmenting the image at the pixel level based on the meaning associated with each region. In embodiments of this disclosure, segmentation refers to the separation of regions corresponding to seated occupants from regions not corresponding to seated occupants, and seated occupant segmentation can be understood as the process of separating the area in the image corresponding to the seated occupant. By utilizing seated occupant segmentation, the area occupied by the seated occupant in the image can be identified more accurately, thereby improving the accuracy of determining the area value of the seated occupant in the image. Such seated occupant segmentation can be performed using an artificial intelligence learning model, which refers to an artificial intelligence algorithm including at least one of machine learning, neural networks, deep learning, classification algorithms, and combinations thereof.

[0121] At the same time, refer to Figure 7The acquired image may contain multiple seats 110. In this case, the controller 150 can determine the area ratio based on the area value of each seat 110 and the corresponding seated occupant, and determine the category of the seated occupant for each seat individually based on this information. For example, in an acquired image including the driver's seat 111 and the front passenger seat 112, when the driver 113 and the front passenger occupant 114 are seated, the controller 150 can independently determine the area values ​​of the driver's seat 111 and the front passenger seat 112, as well as the area values ​​of the seated occupants (i.e., the driver 113 and the front passenger occupant 114). Then, the area ratio of the driver's seat 111 is determined by dividing the area value of the driver 113 by the area value of the driver's seat, and the area ratio of the front passenger seat 112 is determined by dividing the area value of the front passenger occupant 114 by the area value of the front passenger seat. Afterwards, the controller 150 determines the category of the driver 113 and the front passenger occupant 114 based on the determined area ratio of each. (Refer to reference...) Figure 3 Conversely, assuming there is no front passenger 114 in the vehicle, the process of determining the category of driver 113 is performed in the same manner as before. Controller 150 may determine the category of the seated occupant as a specific category, such as "unoccupied" or "empty," in response to no occupant detected in the front passenger seat 112 or the area value of the seated occupant being 0.

[0122] Already referred to Figure 6 and Figure 7 The process of deriving the area values ​​of seat 110 and seated occupants for classifying seated occupants is described, and will be referred to below. Figure 8 Describe the process of determining the category of seated occupants based on derived area values.

[0123] Figure 8 This is a diagram illustrating the classification of seated occupants based on area ratio and threshold according to an embodiment of the present disclosure.

[0124] Figure 8 A graph is shown with one axis representing time and the other axis representing the area ratio. Here, the area ratio refers to the ratio between the area value of seat 110 as described above and the area value of the seated occupant, and the threshold can be understood as a standard for dividing the range of area ratios into different segments. The threshold is a predetermined value that may include one or more thresholds, and can determine the category of the seated occupant for each segment formed by at least one threshold. When using the first to fourth thresholds as examples, such as... Figure 8As shown, five segments are formed on the graph. Therefore, the controller 150 can classify seated occupants into five categories: a first category with an area ratio less than a first threshold, a second category with an area ratio equal to or greater than the first threshold but less than a second threshold, a third category with an area ratio equal to or greater than the second threshold but less than a third threshold, a fourth category with an area ratio equal to or greater than the third threshold but less than a fourth threshold, and a fifth category with an area ratio equal to or greater than the fourth threshold. Assuming the area ratio is defined by dividing the area value of the seated occupant by the area value of the seat, a larger area ratio indicates a larger size of the seated occupant. This means that occupants classified as the first category may represent the smallest individuals or unoccupied states, while occupants classified as the fifth category may represent relatively larger individuals. For example, seated occupants can be classified as shown in Table 1.

[0125] Table 1

[0126] Area ratio < first threshold Category 1 big First threshold ≤ area ratio < second threshold Category 2 medium Second threshold ≤ area ratio < third threshold Category 3 Small Third threshold ≤ area ratio < fourth threshold Category 4 child Fourth threshold ≤ area ratio Category 5 null

[0127] In the following text, reference will be made to Figure 9 The sequence of passenger classification processing according to embodiments of the present disclosure is described. Figure 9 This is a flowchart illustrating the sequence of a method for classifying seated occupants in a vehicle according to an embodiment of the present disclosure.

[0128] Reference Figure 9 In step S901, the image of the vehicle interior captured by camera 140 is first input to controller 150. In step S902, controller 150 also obtains the position of seat 110 from sensors such as seat track position sensor 120, including the forward and backward positions of seat 110 and the seat back tilt angle.

[0129] In step S903, based on the position of the seat 110, the controller 150 determines the area value of the seat 110. In this case, the area value of the seat 110 is determined based on parameters associated with the position of the seat 110, and these parameters can be predetermined to correspond at least to the foremost and rearmost positions of the seat 110. Additionally, the controller 150 can further consider the seat back recline angle in determining the area value of the seat 110, and in this case, these parameters can also be configured to be associated with the seat back recline angle.

[0130] Subsequently, in step S904, the controller 150 detects seated occupants from the acquired image. In this case, when no seated occupant is detected ("No" in step S904), the controller 150 may determine the category of the seated occupant as a specific category, such as the first category, in step S905. When a seated occupant is detected ("Yes" in step S904), the controller 150 determines the area ratio in step S906 based on the area value of the seat 110 and the area value of the seated occupant on the seat 110.

[0131] The controller 150 determines the category of the seated occupant based on the determined area ratio, and in this case, the determination of the seated occupant category can be performed based on a comparison between the determined area ratio and a threshold. The controller 150 may determine the seated occupant's category as the second category in step S908 if the area ratio is equal to or greater than a first threshold but less than a second threshold ("Yes" in step S907); determine the seated occupant's category as the third category in step S910 if the area ratio is equal to or greater than the second threshold but less than a third threshold ("Yes" in step S909); determine the seated occupant's category as the fourth category in step S911 if the area ratio is equal to or greater than the third threshold but less than a fourth threshold ("Yes" in step S911); determine the seated occupant's category as the fifth category in step S914 if the area ratio is equal to or greater than the fourth threshold ("Yes" in step S913); and determine the seated occupant's category as the first category in step S915 if the area ratio is less than the first threshold ("No" in step S913). The process of determining the seated occupant's category based on the comparison between the area ratio and the thresholds can be performed in the order described, but can also be performed individually based on the satisfaction of each condition.

[0132] Although this disclosure has been shown and described in conjunction with specific embodiments, it will be apparent to those skilled in the art that various modifications and alterations may be made therein without departing from the scope of this disclosure as defined by the appended claims.

[0133] Cross-references to related applications

[0134] This application claims priority to Korean Patent Application No. 10-2022-0089106 and Korean Patent Application No. 10-2022-0089107, filed on July 19, 2022, the entire contents of which are incorporated herein by reference.

Claims

1. A seatbelt wearing determination device, the seatbelt wearing determination device comprising: A camera configured to capture images of seated occupants inside the vehicle; as well as A controller configured to determine a first bounding box and a second bounding box from the captured image, and to determine whether the seated occupant is wearing a seatbelt based on the degree of overlap between the first and second bounding boxes. Wherein, the first bounding box corresponds to the upper body area of ​​the seated occupant, and the second bounding box includes the seat belt detected from the captured image, and The controller is further configured as follows: Obtain the seat position of the vehicle's seats; A first area value of the seat is determined based on parameters associated with the obtained seat position of the seat; The seated occupants were detected from the captured images; Determine the second area value of the seated occupant; The category of the seated occupant is determined based on the area ratio between the first area value and the second area value; and The first bounding box is determined based on the identified category of the seated occupant.

2. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to: Detect at least one predetermined point in the upper body region of the seated occupant from the captured image; and The first bounding box is determined based on the detected at least one predetermined point.

3. The seatbelt wearing determination device according to claim 2, wherein, The at least one predetermined point includes a plurality of points corresponding to the center of the shoulders and abdomen of the seated occupant, respectively.

4. The seatbelt wearing determination device according to claim 3, wherein, The controller is also configured to: The width of the first bounding box is determined based on two points from the plurality of points that correspond to the shoulders of the seated occupant, respectively; and The height of the first bounding box is determined based on at least one of the plurality of points.

5. The seatbelt wearing determination device according to claim 2, wherein, The controller is also configured to: In response to the fact that at least one predetermined point is not detected at least partially, the undetected predetermined point is derived from the other detected predetermined points; and The first bounding box is determined based on the exported points.

6. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to: Extract seat belt feature information from the captured images; Classification information is obtained from the extracted seat belt feature information; and The seat belt is detected based on the obtained classification information.

7. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to compare the degree of overlap with a predetermined threshold to determine whether the seated occupant is wearing the seatbelt.

8. The seatbelt wearing determination device according to claim 7, wherein, The controller is also configured to determine that the seated occupant is not wearing the seatbelt in response to the overlap being equal to or less than the predetermined threshold.

9. The seatbelt wearing determination device according to claim 1, wherein, The seat position includes the seat's fore-aft position, and The parameters include a first parameter and a second parameter, which correspond to the foremost and rearmost positions of the seat, respectively.

10. The seatbelt wearing determination device according to claim 1, wherein, The seat position includes the angle at which the seat back is reclined, and The controller is also configured to further determine the first area value of the seat based on the recline angle of the seat.

11. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to determine the category of the seated occupant as a predetermined category in response to the absence of detection of the seated occupant from the captured image.

12. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to: Seated occupant segmentation is obtained from the captured images; and The second area value of the seated occupant is determined based on the obtained seated occupant segmentation.

13. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to compare the area ratio with a predetermined threshold to determine the category of the seated occupant.

14. The seatbelt wearing determination device according to claim 13, wherein, The controller is also configured to determine the category of the seated occupant based on multiple predetermined categories, each with a different range.

15. The seatbelt wearing determination device according to claim 1, wherein, The controller is also configured to obtain the weight information of the seated occupant and determine the category of the seated occupant based on the obtained weight information.

16. The seatbelt wearing determination device according to claim 15, wherein, The controller is also configured to obtain the weight information of the seated occupant from a weight sensor connected to the seat.

17. The seatbelt wearing determination device according to claim 1, wherein, The vehicle includes multiple seats, and The controller is also configured to determine the area ratio of each seat and, based on the area ratio determined for each seat, to determine the category of the seated occupant for each seat.

18. The seatbelt wearing determination device according to claim 1, wherein, In order to determine at least one of the first bounding box and the second bounding box, the controller is configured to input the captured image into an artificial intelligence algorithm, which includes at least one of machine learning, neural networks, deep learning, and classification algorithms.

Images