Method for identifying seat occupancy in a vehicle

Abstract

Method (30) for identifying a seat occupancy in a vehicle (20), comprising the steps, Receiving surveillance data from a camera (22) of a vehicle interior (24), Assigning pixels of the monitoring data to components of the vehicle (20) by means of semantic segmentation using a first neural network, Identifying one or more persons in the surveillance data using a second neural network to recognize a person's pose, Combining the assigned pixels to the vehicle components (20) with the detected pixels of one or more persons to identify a seat occupancy in the vehicle (20).

Description

State of the art

[0001] From EP 1 746 527 A2 an occupant information and detection system, occupant restraint system, and vehicle are known. Disclosure of the invention

[0002] A method for identifying a seat occupancy in a vehicle is proposed, comprising the steps of Receiving surveillance data from a camera of a vehicle interior, assigning pixels of the surveillance data to components of the vehicle by means of semantic segmentation using a first neural network, detecting one or more persons in the surveillance data by means of a second neural network to detect a person's pose, merging the assigned pixels to the components of the vehicle with the detected one or more persons to identify a seat occupancy in the vehicle.

[0003] The advantage of the present invention is that occupancy of the vehicle seats can be identified in a particularly safe and / or reliable manner. Combining can, for example, be carried out or designed as a superimposition of the detected components and persons. This makes it possible to detect seat occupancy in the vehicle. In other words, it can be detected whether there are people in the vehicle and, if people have been detected in the vehicle, where they are located or where they have taken their seats. This makes it possible to increase safety in the vehicle because, for example, safety functions can be triggered in a targeted manner. Furthermore, comfort can be increased because, for example, comfort functions can be adapted to the people in the vehicle.

[0004] This method can advantageously be used in any vehicle in which a number of people and / or a seat assignment is to be identified. Advantageously, seat assignment can be enabled using the proposed method by using the various steps of component assignment and person recognition using a neural network, in particular without calibration, for any vehicle type and camera installation position. This can be used, for example, for a seat belt warning or statistics on seat occupancy or other surveys in driving services. Furthermore, the method can be used, in particular, to detect when a person moves away from their seat during a journey.Furthermore, for some devices in a vehicle, it may be relevant whether a person is in the driver's seat and therefore whether the car is occupied by a driver or not, for example, during manual takeover. This can increase safety in the vehicle, especially in road traffic.

[0005] In an exemplary embodiment, in the step of detecting one or more persons in the surveillance data, a two-dimensional pose of a person or a three-dimensional pose of a person can be detected. This allows, in particular, a person's pose and thus the presence of a person in the vehicle to be determined safely and / or reliably.

[0006] Advantageously, in the step of detecting one or more persons in the surveillance data, the height of one or more persons can be detected. This makes it possible, in particular, to determine the distance the person is from a camera. This makes it possible, for example, to identify whether the person is sitting in the front or rear seats.

[0007] In a further development, in the step of detecting one or more persons in the monitoring data, the joint points of one or more persons can be detected. This allows, in particular, a pose of one or more persons to be detected safely and / or reliably, which allows, in particular, a person's presence in the vehicle to be determined safely and / or reliably.

[0008] In an advantageous embodiment, the second neural network for recognizing a person's pose can be designed as a trained neural network. Alternatively or additionally, the first neural network for semantic segmentation can be designed as a trained neural network. This allows the method to be particularly robust, allowing a person in the vehicle to be recognized safely and / or reliably and / or the pixels to be assigned to a component in the vehicle.

[0009] Preferably, in the pixel assignment step, the monitoring data can be assigned to vehicle components on a pixel-by-pixel basis per frame. This can enable a secure and / or reliable assignment of the pixels to the vehicle components.

[0010] In an exemplary embodiment, in the step of assigning pixels, pixels can be assigned to one or more components of the vehicle that are arranged in front of a person, wherein in the step of merging it is recognized that the person is arranged behind the one or more components of the vehicle. Alternatively or additionally, in the step of assigning pixels, pixels can be assigned to a door of the vehicle that is arranged in front of a person, wherein in the step of merging it is recognized that the person is arranged behind the door outside the vehicle. Alternatively or additionally, in the step of assigning pixels, pixels can be assigned to a vehicle seat that is arranged behind a person, wherein in the step of merging it is recognized that the vehicle seat is arranged behind the person and the person is arranged on the vehicle seat.In particular, this makes it possible to safely and / or reliably identify whether people are located in front of or behind a component and / or inside or outside a vehicle.

[0011] In a further development, in the pixel assignment step, the pixels can be assigned to a vehicle seat pixel class to identify which vehicle seat the pixels are assigned to. This allows, in particular, to reliably and safely identify which vehicle seat, for example, the driver's seat, the front passenger seat, or the rear seat bench, a person is positioned or has taken a seat.

[0012] Furthermore, a system for identifying a seat occupancy in a vehicle is proposed, wherein the system is designed to carry out a method for identifying a seat occupancy in a vehicle.

[0013] In an advantageous embodiment, the system comprises a camera for recording surveillance data of a vehicle interior. This ensures, in particular, the secure and / or reliable provision of surveillance data of a vehicle interior.

[0014] Furthermore, a method is proposed for training a first neural network for assigning pixels of the monitoring data to components of the vehicle by means of a semantic segmentation for use in a method for identifying a seat occupancy in a vehicle, comprising the steps Receiving surveillance data from a camera of a vehicle interior, learning for each pixel in a frame to which component of the vehicle the pixel is assigned. Using a trained neural network, the pixels can be assigned reliably and / or to a component in the vehicle.

[0015] Furthermore, a method is proposed for training a second neural network for recognizing a pose of a person for use in a method for identifying a seat occupancy in a vehicle, comprising the steps of receiving monitoring data from a camera, Learning poses of multiple people. A trained neural network can be used to detect a person in the vehicle safely and / or reliably. Short description of the drawings

[0016] Embodiments of the invention are illustrated in the drawings and explained in more detail in the following descriptions. The same reference numerals are used for the elements shown in the various figures and have a similar effect, and a repeated description of the elements is omitted. They show: Fig. 1 is a schematic representation of a vehicle with a device for observing a vehicle occupant; Fig. 2 a schematic representation of a method according to an embodiment of the present invention; Fig. 3 is a schematic representation of a method according to an embodiment of the present invention; Fig. 4 a schematic representation of a method according to an embodiment of the present invention; Fig. 5 is a schematic diagram of a system according to an embodiment of the present invention; Fig. 6 is a schematic representation of a vehicle interior according to an embodiment of the present invention; Fig. 7 is a schematic representation of a vehicle interior according to an embodiment of the present invention. Embodiments of the invention:

[0017] Fig. 1 shows a schematic representation of a vehicle 20, for example a motor vehicle, for example a car, with an observation device 22. The vehicle 20 has, in particular, an interior 24 or a vehicle interior 24, wherein in particular one or more seats 26 for one or more vehicle occupants 28 can be arranged in the vehicle interior 24. Furthermore, the vehicle 20 has the observation device 22 for observing the vehicle interior 24 and / or one or more vehicle occupants 28, for example a driver and / or a front passenger and / or one or more further passengers. The observation device 22 can also be referred to as an observation system, driver monitoring system, occupant observation system and / or a monitoring system. The observation device 22 can, for example, be designed to monitor a direction of gaze, a body posture and / or the position of the head or face orthe eyes of the vehicle occupant 28 or the fatigue level and / or other vital signs of the vehicle occupant 28. Furthermore, for example, an identity of the vehicle occupant 28 can be recorded.

[0018] The observation device 22 can be arranged in particular in or on a dashboard, in or on an instrument panel, in or on a steering wheel, in or on a vehicle roof, on a window, in or on a rearview mirror or in or on a pillar, for example on an A-pillar and / or a B-pillar of the vehicle 20, or at another location in the vehicle 20.

[0019] To observe the vehicle occupant 28, the observation device 22 can have a recording unit or be designed as a recording unit. Advantageously, the observation device 22 can have a camera or be designed as a camera. The camera can, in particular, be directed toward the vehicle interior 24.

[0020] In a further development, the observation device 22 can have a lighting unit for emitting light rays, in particular infrared rays. The lighting unit can, for example, be directed towards the vehicle interior 24 in order to illuminate it with light rays, for example infrared rays. The lighting unit can, for example, be designed as a lighting unit, lighting element, light-emitting diode, LED, OLED and / or laser diode and / or can have a lighting unit, a lighting element, a light-emitting diode, an LED, OLED and / or a laser diode. The light rays emitted by the lighting unit can preferably be reflected on or in the vehicle interior, wherein the reflected light rays can be directed in the direction of the recording unit. Alternatively, the observation device 22 can not have a separate lighting unit and the vehicle interior orthe vehicle occupants are illuminated by light from an environment.

[0021] The recording unit can be designed, for example, as an image recording unit, for example, as an optical sensor or a camera, in particular as an infrared camera module, wherein the recording unit is directed toward the vehicle interior 24 in order to visually capture it. The design as an infrared camera module makes it possible to conduct the observation even at night, without the vehicle occupant 28 being brightly illuminated and thus, in particular, disturbed or blinded.

[0022] Furthermore, the observation device 22 has a control unit 29 or an evaluation unit 29 or a computing unit 29 for controlling the lighting unit and / or the recording unit and / or for processing the data recorded by the recording unit. The control unit 29 can, in particular, be part of the observation device 22.

[0023] Fig. Fig. 2 shows a schematic representation of a method 30 for identifying a seat occupancy in a vehicle according to an embodiment of the present invention. The method 30 can be implemented by means of a system according to the system according to Fig. 5 are executed.

[0024] In a first step 32 of the method 30, monitoring data from a camera of a vehicle interior are received. The monitoring data can, for example, be recorded by a camera in the vehicle interior. The camera can, for example, be designed as an observation device or as an observation unit and be arranged in the vehicle. The observation device can, for example, be configured according to the observation device according to Fig. 1 be trained.

[0025] In a second step 34 of the method 30, pixels of the monitoring data are assigned to components of the vehicle by means of semantic segmentation using a first neural network. Preferably, the first neural network for semantic segmentation can be designed as a trained neural network. An application of semantic segmentation can, for example, be carried out according to Fig. 7 are shown.

[0026] In an advantageous embodiment, the monitoring data can be assigned to vehicle components on a pixel-precise basis for each frame. In other words, each pixel in a frame can be assigned to a component of the vehicle. Components can be, for example, a vehicle seat, a door, a window, a tailgate, a steering wheel, a pillar, or other elements in the vehicle. In a further development, the pixels can be assigned to a vehicle seat pixel class to identify which vehicle seat the pixels are assigned to. Advantageously, it can be identified which vehicle seat is involved, for example a driver's seat, a front passenger seat, or a rear seat, for example the right rear seat, the left rear seat, or the middle rear seat. The same applies to all other rear seats, provided they are within the field of view of step 32.

[0027] In other words, semantic segmentation can be performed using the first network. This allows, in particular, pixel-precise classification of the image.

[0028] In a further development, the determined segmentation areas or recognized areas of the semantic segmentation can be aggregated into a static mask in a further step. In other words, a mask can be created using the recognized components or areas of the vehicle interior. In particular, all non-static elements, such as people and objects, are ignored and only static elements are aggregated over time. This process creates a static interior mask. This step can be performed once at a previously defined time and / or when no non-static elements, such as a person, are in the vehicle. The further step can advantageously be performed after or before the second step 34.

[0029] Advantageously, it can be determined which parts of the vehicle are in front of a person, for example a front seat for a person sitting in the back seat or in the second row. Furthermore, a door pixel, for example, can be recognized or assigned. If it is determined that the door pixel is located in front of a recognized person, it can be concluded that the person is outside the vehicle. Advantageously, a seat pixel can be recognized. In particular, a seat pixel obscured by a person pixel can indicate that the seat is located behind the recognized person. Furthermore, a seat pixel class can be determined, whereby the seat pixel class can be used to determine which type of vehicle seat is involved. For example, it can be a driver's seat, a front passenger seat, a right, left or middle seat in the back seat.

[0030] In a third step 36 of the method 30, one or more persons are recognized in the surveillance data by means of a second neural network for recognizing a person's pose. Preferably, the second neural network for recognizing a person's pose can be designed as a trained neural network. The third step 36 can advantageously be performed after, before, or parallel to the second step 34. A person or a person's pose can be recognized, for example, according to Fig. 6 are shown.

[0031] In a further development, a two-dimensional pose of a person or a three-dimensional pose of a person can be recognized. For this purpose, the monitoring data can be configured, for example, as two-dimensional monitoring data or as three-dimensional monitoring data, which are recorded or provided by a two-dimensional monitoring unit or a three-dimensional monitoring unit.

[0032] In an advantageous embodiment, the height of one or more persons can be detected. This makes it possible to detect, in particular, whether a person is in the front area of ​​the vehicle, i.e., in the front seats, or in the back seat. If the height of an adult is smaller than that of another adult in the vehicle, it can be assumed that the person who was detected as smaller is seated further back. Furthermore, joint points of one or more persons can advantageously be detected. For example, an elbow joint, a shoulder joint, a wrist, a neck, or others can be formed or detected as joint points. Detection of joint points of persons can, for example, be carried out according to Fig. 6 are shown.

[0033] In other words, a person's pose can be determined using the second network. This allows the person's 2D position in the image and / or a 3D pose to be determined. Advantageously, the approximate size of the person in the image can be determined, which makes it possible, in particular, to determine which seat row a person can be assigned to. If it is detected that a person is very small compared to others and, for example, is mostly positioned within a window, then it can be concluded that the person is outside the car.

[0034] In a fourth step 38 of the method 30, the detected one or more persons are merged with the associated pixels to the components of the vehicle to identify a seat occupancy in the vehicle. A merge can be carried out, for example, according to Fig. 7 are shown.

[0035] In other words, the relationship and / or arrangement of the detected person(s) is / are relative to the vehicle components. This makes it possible to identify where the detected person(s) are positioned in front of, on, or behind the vehicle components. This makes it possible to determine whether the detected person(s) are positioned behind a vehicle seat, in front of a vehicle seat, or behind a vehicle door. In other words, it is possible to identify whether the detected person(s) are seated on the vehicle seat and which vehicle seat this is.

[0036] One way to assign a seat to a person is to calculate the common intersection area between the area resulting from the parts of the pose, especially without considering head support points, and the area of ​​the seat. In a further development, the assignment of areas to areas or points to areas can be done using a mathematical procedure. This can be done, for example, using the procedure according to Fig. 8 are shown.

[0037] In a further development, the merging can advantageously be carried out by means of a neural network, in particular another neural network.

[0038] In an exemplary embodiment, pixels can in particular be assigned to one or more components of the vehicle that are arranged in front of a person, wherein the merging step detects that the person is arranged behind the one or more components of the vehicle. For example, it can be detected that a vehicle seat is arranged in front of a person. This makes it possible to detect that the person is located behind the vehicle seat. For example, the vehicle seat can be the driver's seat, with a person being identified behind the driver's seat. This makes it possible to detect that the person is arranged or has taken a seat behind the driver's seat.

[0039] In a further embodiment, pixels can be assigned, for example, to a vehicle seat located behind a person, wherein the merging step detects that the vehicle seat is located behind the person and that the person is positioned on the vehicle seat. In other words, it detects that the person has taken a seat or is sitting on the vehicle seat.

[0040] In a further embodiment, pixels can be assigned, for example, to a vehicle door located in front of a person, wherein the merging step detects that the person is located behind the door outside the vehicle. Alternatively or additionally, pixels can be assigned, for example, to a vehicle window, wherein the detection is that the person is located inside the window, in particular only inside the window. This makes it possible to detect that the person is outside the vehicle and therefore not inside it.

[0041] In other words, the method is based on the output of two networks. The first network can be configured for semantic segmentation or object detection. The second network can be configured as a person detector and / or a person pose detector and / or a 3D person pose detector.

[0042] Advantageously, the two networks can be trained. The first neural network can, for example, be trained according to the method according to Fig. 3. The second neural network can be trained, for example, according to the method according to Fig. 4. The semantic segmentation learns specifically for each pixel in the image, whether it is, for example, a window, a left seat in the second row, a seat back, a person, or other components or areas. Alternatively or additionally, object detection can be trained for these classes. The second network trains a person pose. The semantic segmentation also has person pixels. However, overlapping people can advantageously be differentiated from one another using a person pose pixel. In other words, with the help of person pose detection, it can be detected when two people are arranged one behind the other or in front of each other and thus overlap.By detecting a person's pose, the individual joint points of the person can be advantageously detected, which allows the distances and positions of the joint points to be determined largely independently of possible occlusion and / or body posture. An example of the detection of joint points can be seen in a step-by-step manner according to . Fig. 6. This advantageously allows for better localization of people in space, especially in the vehicle interior. In a further development, the second network can be configured as a 3D pose detector. In this case, the joint positions in space can be assigned absolutely or relative to the person's center. Alternatively or additionally, object detection for people can be used.

[0043] By combining the two networks, seat assignment can advantageously be determined. Using pixel-precise semantic segmentation, it can be determined whether a person is behind a seat or sitting on the seat. Furthermore, it can be determined whether a detected person is inside or outside the vehicle. Using semantic segmentation, for example, window pixel information can be used to determine whether a pose is in the window area. This can be further supported by deliberately training visible person pixels outside the vehicle as window pixels. This allows the pixel class to be used in addition to the geometric derivation to determine whether a person is inside or outside the vehicle.

[0044] In other words, the described method can be used to combine the output of two neural networks to enable seat assignment for any vehicle. For this purpose, the first neural network can be used to determine which pixels belong to which areas of the vehicle, for example, windows, rear left seat, front right headrest, and others. The second neural network can be used to determine a person's pose. In combination, it can be determined, in particular for any vehicle and / or any camera installation position, which seat a detected person or several detected people are located in. This can also be used to determine how many people are in the vehicle. In other words, the number of people in the vehicle can be determined.The method can advantageously perform a seat assignment, in particular without calibration, for any vehicle type and / or camera installation position.

[0045] The method can be used in any vehicle in which the number of people and seat assignment needs to be determined, and can therefore be useful. The method can, for example, have an additional step for a seat belt warning. In other words, a detected person can be warned if they are not wearing a seat belt. In a further development, statistics on seat occupancy in ride-hailing services can be collected. Furthermore, it can be detected when a person moves away from a seat during a journey. Furthermore, for some components or devices in the car, it is relevant whether a person is in the driver's seat and therefore whether the car has a driver or not. This can be relevant both for safety reasons and for statistical purposes.

[0046] Advantageously, the surveillance data recorded by a camera can be surveillance data from a vehicle interior. The camera can be located inside the vehicle interior for this purpose. Advantageously, the camera can be permanently installed in the vehicle. Alternatively or additionally, the camera can be retrofitted in the vehicle, for example, as a retrofittable dashcam.

[0047] In other words, by combining the two networks in the method, a seat assignment can be carried out or determined for any vehicle class and / or camera installation position.

[0048] Fig. 3 shows a schematic representation of a method 40 for training a first neural network for assigning pixels of the monitoring data to components of the vehicle by means of semantic segmentation for use in a method for identifying a seat occupancy in a vehicle according to an embodiment of the present invention. The method for identifying a seat occupancy in a vehicle can, for example, be carried out according to the method according to Fig. 2 be trained.

[0049] In a first step 42 of the method 40, monitoring data from a camera of a vehicle interior are received. In a second step 44 of the method 40, for each pixel in a frame, it is learned to which component of the vehicle the pixel is assigned

[0050] Fig. 4 shows a schematic representation of a method 46 for training a second neural network for recognizing a pose of a person for use in a method for identifying a seat occupancy in a vehicle according to an embodiment of the present invention. The method for identifying a seat occupancy in a vehicle can, for example, be carried out according to the method according to Fig. 2 be trained.

[0051] In a first step 48 of the method 46, surveillance data from a camera is received. In a second step 50 of the method 46, poses of several people are learned.

[0052] Fig. 5 shows a schematic representation of a system 52 for identifying a seat occupancy in a vehicle according to an embodiment of the present invention. The system 52 is designed to carry out a method for identifying a seat occupancy in a vehicle 20 according to Fig. 2. The vehicle can be used according to vehicle 20 according to Fig. 1 be trained.

[0053] To carry out the method, the system 52 may have a computing unit 54. The computing unit 54 may be arranged outside the vehicle 20 or inside the vehicle 20. In this advantageous embodiment, the system 52 further comprises a camera 22. The camera 22 may, for example, be designed as an observation device 22. The observation device 22 or the camera 22 may, according to Fig. 1 and / or arranged in the vehicle 20.

[0054] Fig. Fig. 6 shows a schematic representation of a vehicle interior 24 according to an embodiment of the present invention. The vehicle interior 24 can be used, for example, in a vehicle according to the vehicle according to Fig. 1 and / or Fig. 5 may be arranged. Fig. 6 shows a representation of a pose output of detected persons in the vehicle interior 24. The detection of the pose can, for example, be carried out according to the method according to Fig. 2. In this advantageous embodiment, a first person 56 is recognized who is positioned in the driver's seat, a second person 58 who is seated in the front passenger seat, a third person 60 who is positioned in the second row behind the driver's seat, and a fourth person 62 who is positioned in the right-hand seat in the third row, thus in particular two seats behind the person in the driver's seat. Advantageously, the joint points for each of the persons can be represented as points and / or the limbs as lines. Advantageously, two shoulder joints 64, two elbow joints 66, a neck 68, and in particular a wrist 70 can be recognized for each recognized person. Furthermore, the upper arm 72 of a person, for example, can be represented by a straight line. This makes it possible to determine that the people have taken their seats.Furthermore, it can be determined that there are four persons in the vehicle 20 according to . Fig. 6 are located.

[0055] Fig. Fig. 7 shows a schematic representation of a vehicle interior 24 according to an embodiment of the present invention. The vehicle interior 24 can be used, for example, in a vehicle according to the vehicle according to Fig. 1 and / or Fig. 5 may be arranged. Fig. Figure 7 shows a representation of semantic segmentation and a pose output of detected persons in the vehicle interior 24. By means of the semantic segmentation, components can be detected in the vehicle interior 24. Advantageously, the pixels of the representation can be Fig. 7 can be assigned to the components of the vehicle interior 24. In an advantageous embodiment, the vehicle interior 24 can alternatively or additionally be divided into different areas. Preferably, the components or areas can be displayed or highlighted in different colors or shades. In this advantageous embodiment, pixels are assigned to a backrest 74 of the driver's seat, a headrest 76 of the driver's seat, a backrest 78 of the front passenger seat, and a headrest 80 of the front passenger seat. Furthermore, pixels can be assigned to one or more windows 82. Furthermore, pixels can be assigned to one or more vehicle pillars 84. Furthermore, pixels can be assigned to the seats 86 of the rear bench and the headrests 88 of the seats in the rear bench. In this advantageous embodiment, pixels can in particular also be assigned to a child seat 90 on the rear bench.

[0056] In this advantageous embodiment, a first person 56 is also detected, who is positioned or has taken a seat in the driver's seat. Furthermore, a pose of the person 56 is detected. The detection of the person 56 and the pose of the person 56 can be carried out, for example, by means of the method according to Fig. 2. A representation of a recognition of a pose of the person 56 can, for example, be determined according to Fig. 6. Advantageously, the joint points for the person 56 in the driver's seat can be represented as points and / or the limbs as lines. Advantageously, shoulder joints 64, elbow joints 66, and others can be recognized. Furthermore, the person's upper arm 72, for example, can be represented by a straight line. This makes it possible to determine that the person has taken a seat, in particular the driver's seat. Furthermore, it can be determined that a person is in the vehicle 20 according to Fig. 7 are located.

[0057] Fig. Fig. 8 shows a schematic representation of a method 30 for identifying a seat occupancy in a vehicle according to an embodiment of the present invention. The method 30 can be implemented by means of a system according to the system according to Fig. 5. The method 30 is carried out according to the method 30 according to Fig. 2. The procedure 30 according to Fig. 8 thus comprises the first step 32, the second step 34, the third step 36 and the fourth step 38. According to Fig. 8, for example, shows representations or images of the vehicle interior according to the processing in the respective steps.

[0058] Advantageously, it is shown that the vehicle interior is imaged in the first step 32. In other words, monitoring data of the vehicle interior is recorded. Thus, the vehicle interior is imaged in the first step. Advantageously, each image can have a defined number of pixels.

[0059] The monitoring data is processed in the second step 34 such that pixels of the monitoring data are assigned to components of the vehicle by means of semantic segmentation using a first neural network. Thus, in the second step, the components of the vehicle interior are categorized and displayed. In an advantageous embodiment, moving objects, such as people, are grayed out or blackened in this step and thus not viewed. The assignment of the components can be carried out according to Fig. 7 be trained.

[0060] In a further development, the determined segmentation regions or recognized regions of the semantic segmentation can be aggregated into a static mask in a further step 35. In other words, a mask of the vehicle interior is created with the various components or regions. In particular, moving objects are ignored.

[0061] In the third step 36 of the method 30, one or more persons are recognized in the surveillance data by means of a second neural network for recognizing a person's pose. A person or a person's pose can be recognized, for example, according to Fig. 6. In particular, the person's joint points can be detected, which allows the person's pose to be inferred. In the third step 36, the vehicle interior is thus displayed with the person and the detected joint points.

[0062] In a fourth step 38 of the method 30, the detected one or more persons are merged with the assigned pixels to the components of the vehicle to identify a seat occupancy in the vehicle. In other words, the results of the second step 34 and the third step 36 can be merged. This results in a representation of the vehicle interior divided into components and a schematically represented person located therein, wherein the person's joint points and thus a recognized pose are displayed. This makes it possible to recognize which seat the person is sitting on. A merge to identify a seat occupancy in the vehicle can, for example, be carried out according to Fig. 6 are shown.

Claims

[1] Method (30) for identifying a seat occupancy in a vehicle (20), comprising the steps of Receiving monitoring data from a camera (22) of a vehicle interior (24), Assigning pixels of the monitoring data to components of the vehicle (20) by means of semantic segmentation using a first neural network, Detecting one or more persons in the surveillance data using a second neural network to detect a person’s pose, Merging the pixels assigned to the components of the vehicle (20) with the detected one or more persons to identify a seat occupancy in the vehicle (20). [2] The method (30) of claim 1, wherein in the step of detecting one or more persons in the surveillance data, a two-dimensional pose of a person or a three-dimensional pose of a person is detected. [3] Method (30) according to one of the preceding claims, wherein in the step of recognizing one or more persons in the monitoring data, a size of the one or more persons is recognized. [4] Method (30) according to one of the preceding claims, wherein in the step of recognizing one or more persons in the monitoring data, articulation points of the one or more persons are recognized. [5] Method (30) according to one of the preceding claims, wherein the first neural network for semantic segmentation is designed as a trained neural network. [6] Method (30) according to one of the preceding claims, wherein the second neural network for recognizing a pose of a person is designed as a trained neural network. [7] Method (30) according to one of the preceding claims, wherein in the step of assigning pixels, a pixel-accurate assignment of the monitoring data to components of the vehicle (20) per frame takes place. [8] Method (30) according to one of the preceding claims, wherein in the step of assigning pixels, pixels are assigned to one or more components of the vehicle (20) which are arranged in front of a person, wherein in the step of merging it is recognized that the person is arranged behind the one or more components of the vehicle (20). [9] Method (30) according to one of the preceding claims, wherein in the step of assigning pixels, pixels are assigned to a door of the vehicle (20) which are arranged in front of a person, wherein in the step of merging it is recognized that the person is arranged behind the door outside the vehicle (20). [10] Method (30) according to one of the preceding claims, wherein in the step of assigning pixels, pixels are assigned to a vehicle seat which is arranged behind a person, wherein in the step of merging it is recognized that the vehicle seat is arranged behind the person and the person is arranged on the vehicle seat. [11] Method (30) according to one of the preceding claims, wherein in the step of assigning pixels, the pixels can be assigned to a vehicle seat pixel class for identifying which vehicle seat the pixels are assigned to. [12] System (52) for identifying a seat occupancy in a vehicle (20), wherein the system is designed to carry out a method (30) according to one of the preceding claims. [13] System (52) according to claim 12, comprising a camera (22) for recording monitoring data of a vehicle interior (24). [14] Method (40) for training a first neural network for assigning pixels of the monitoring data to components of the vehicle (20) by means of semantic segmentation for use in a method for identifying a seat occupancy in a vehicle (20) according to one of the preceding claims 1 to 11, comprising the steps of Receiving monitoring data from a camera (22) of a vehicle interior (24), Learning for each pixel in a frame to which component of the vehicle (20) the pixel is assigned. [15] Method (46) for learning a second neural network for recognizing a pose of a person for use in a method for identifying a seat occupancy in a vehicle (20) according to one of the preceding claims 1 to 11, comprising the steps of receiving monitoring data from a camera (22), learning poses of a plurality of persons.

Images