Method, device and vehicle for detecting when a vehicle occupant's body part is placed out of a window

Abstract

The present disclosure relates to a method, device, and vehicle for detecting whether a body part of a vehicle occupant has been exposed through a window, the method including: detecting whether a body part of a vehicle occupant has been exposed through a window based on at least two types of sensing data, obtaining at least two types of detection results; specifying a determination result based on the at least two types of detection results; and generating warning information when the determination result indicates that a body part of the occupant has been exposed through a window. According to an embodiment of the present disclosure, the image data and radar sensing data are combined to specify the determination result and generate the warning information, which is advantageous for improving the accuracy of the determination result and improving the safety of occupants in the vehicle cabin.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application, application number 202111273194.9, filed with the China Patent Office on October 29, 2021, entitled "Method, device and vehicle for detecting when a body part of a vehicle occupant has been pushed out of a window," the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to the field of computers, and more particularly to a method, an apparatus, and a vehicle for detecting when a body part of a vehicle occupant has been placed out of a window. [Background technology]

[0003] When a driver or passenger sticks their limbs out of a window while driving a vehicle, it is a very dangerous act, which threatens driving safety and is likely to cause a traffic accident. Therefore, how to accurately detect when a driver or passenger sticks their limbs out of a window and respond in a timely manner is of great importance to the safety of the driver and passengers in the vehicle. Summary of the Invention

[0004] The present disclosure proposes a technical solution for detecting when a body part of a vehicle occupant is sticking out of a window.

[0005] According to one aspect of the present disclosure, based on at least two types of sensing data including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting obstacles in the window area, detecting whether or not a body part of a vehicle occupant has stuck out of a window, and obtaining at least two systems of detection results; determining a determination result based on the at least two detection results; and generating warning information when the determination result indicates that the body part of the vehicle occupant has stuck out the window.

[0006] In one possible implementation, the method comprises: The method further includes acquiring vehicle body signal data including door data indicating an open / close state of a door and window data indicating an open / close state of a window; Detecting whether or not a body part of a vehicle occupant has stuck out a window based on each of the at least two types of sensing data described above and obtaining at least two systems of detection results is When the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the passenger is open, the method detects whether or not a body part of the passenger has protruded from the window based on each of at least two types of sensing data of the vehicle, and obtains the detection results of the at least two systems.

[0007] In one possible implementation, when the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the vehicle occupant is open, detecting whether or not a body part of the vehicle occupant has protruded out of the window based on each of at least two types of sensing data, and obtaining at least two systems of detection results, Transmitting a trigger signal to the radar sensor to collect sensing data, and acquiring the radar sensing data collected by the radar sensor; acquiring image data collected by the on-board camera; Detecting whether or not a body part of the occupant has protruded from a window corresponding to the seating position of the occupant based on the radar sensing data and the image data, respectively, and obtaining a detection result based on the radar sensor and a detection result based on the image data.

[0008] In one possible implementation, the method comprises: The method further includes determining, as the determination result, that no body part of the occupant is sticking out of a window when the vehicle body signal data indicates that a door is open or that all windows corresponding to the occupant's seating position are closed.

[0009] In one possible implementation, detecting whether a body part of a vehicle occupant has stuck out a window based on each of the at least two types of sensing data and obtaining at least two systems of detection results is: Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; obtaining a second detection result based on the radar sensing data; The step of determining the determination result based on the at least two detection results includes: The method includes determining a determination result based on the first detection result and the second detection result.

[0010] The step of determining a determination result based on the first detection result and the second detection result includes: When both the first detection result and the second detection result indicate that a body part of the occupant has been stuck out of a window, specifying as the determination result that the body part of the occupant has been stuck out of a window; When the first detection result indicates that a body part of the passenger has been put out of a window and the second detection result indicates that the body part has not been put out of a window, determining that the body part of the passenger has not been put out of a window as the determination result; and when the first detection result indicates that a body part has not been put out through the window and the second detection result indicates that a body part has been put out through the window, determining as the determination result that the body part of the occupant has been put out through the window.

[0011] In one possible implementation, detecting whether a body part of a vehicle occupant has stuck out a window based on each of the at least two types of sensing data and obtaining at least two systems of detection results is: Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; When the first detection result indicates that a body part has been stuck out of a window, acquiring seat information of a target passenger who has stuck out a body part in the image data through a window based on the first detection result; Obtaining the radar sensing data of a target window corresponding to seat information of the target passenger; and identifying a second detection result based on the radar sensing data of the target window.

[0012] In one possible implementation form, the acquiring of the radar sensing data of the target window corresponding to the seat information of the target occupant includes: acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant, and vehicle body signal data related to the seat information of the target occupant, including first signal data indicating an open / close state of a target door that is a door corresponding to the seat information of the target occupant, and second signal data indicating an open / close state of the target window; Identifying the second detection result based on the radar sensing data includes: The method includes identifying the second detection result based on the radar sensing data and the vehicle body signal data.

[0013] In one possible implementation mode, determining a judgment result based on the at least two systems of detection results includes: When the second detection result indicates that the body part of the occupant has stuck out the window, the determination result includes specifying that the body part of the occupant has stuck out the window.

[0014] In one possible implementation mode, the step of recognizing a relative state between a body part of a passenger and a window in the image data and obtaining a first detection result includes: The method includes recognizing a relative state between a body part of a vehicle occupant and a window based on the image data, and obtaining a degree of confidence that the body part of the vehicle occupant has been pushed out of the window.

[0015] In one possible implementation mode, determining a judgment result based on the at least two systems of detection results includes: If the reliability is greater than a predetermined threshold and the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; If the reliability is smaller than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; The method includes at least one of: when the reliability is smaller than a predetermined threshold value; when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window; and when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window, identifying the body part of the vehicle occupant as not sticking out of the window as the judgment result.

[0016] In one possible implementation, the vehicle-mounted camera includes a camera installed inside the vehicle cabin and / or a camera installed outside the vehicle cabin.

[0017] In one possible implementation, the radar sensors are provided at the top / bottom edges of the window.

[0018] In one possible implementation, the sensing data further includes infrared sensing data and / or millimeter wave sensing data.

[0019] According to one aspect of the present disclosure, there is provided a detection module for detecting whether a body part of a vehicle occupant has protruded from a window based on at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in a window area, and for obtaining at least two types of detection results; an identification module for identifying a determination result based on the at least two detection results; and a warning module for generating warning information when the determination result indicates that the body part of the occupant has been protruded from the window.

[0020] In one possible implementation, the device comprises: further comprising a vehicle body signal data acquisition module for acquiring vehicle body signal data including door data indicating an open / closed state of a door and window data indicating an open / closed state of a window; When the vehicle body signal data indicates that the door is closed and the window corresponding to the passenger's seating position is open, the detection module is used to detect whether the passenger's body part has protruded out the window based on each of at least two types of sensing data from the vehicle, and to obtain the at least two systems of detection results.

[0021] In one possible implementation, if the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of a passenger is open, the detection module: Transmitting a trigger signal to the radar sensor to collect sensing data, and acquiring the radar sensing data collected by the radar sensor; acquiring image data collected by the on-board camera; It is used to detect whether or not a body part of the occupant has protruded from a window corresponding to the occupant's seating position based on the radar sensing data and the image data, respectively, and to obtain detection results based on the radar sensor and detection results based on the image data.

[0022] In one possible implementation, the device is further used to determine, as the determination result, that the occupant's body part is not sticking out of the window when the vehicle body signal data indicates that a door is open or that all windows corresponding to the occupant's seating position are closed.

[0023] In one possible implementation, the detection module comprises: Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; and obtaining a second detection result based on the radar sensing data; The specific module is The determination result is determined based on the first detection result and the second detection result.

[0024] In one possible implementation, the specific module: When both the first detection result and the second detection result indicate that a body part of the occupant has been stuck out of a window, specifying as the determination result that the body part of the occupant has been stuck out of a window; When the first detection result indicates that a body part of the passenger has been put out of a window and the second detection result indicates that the body part has not been put out of a window, determining that the body part of the passenger has not been put out of a window as the determination result; When the first detection result indicates that a body part has not been put out through the window and the second detection result indicates that a body part has been put out through the window, the determination result is used to at least one of identify that the body part of the occupant has been put out through the window.

[0025] In one possible implementation, the detection module comprises: Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; When the first detection result indicates that a body part has been stuck out of a window, acquiring seat information of a target passenger who has stuck out a body part in the image data through a window based on the first detection result; Obtaining the radar sensing data of a target window corresponding to seat information of the target passenger; and determining a second detection result based on the radar sensing data of the target window.

[0026] In one possible implementation form, the acquiring of the radar sensing data of the target window corresponding to the seat information of the target occupant includes: acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant, and vehicle body signal data related to the seat information of the target occupant, including first signal data indicating an open / close state of a target door that is a door corresponding to the seat information of the target occupant, and second signal data indicating an open / close state of the target window; The step of identifying the second detection result based on the radar sensing data of the target window includes: The method includes identifying the second detection result based on the radar sensing data and the vehicle body signal data.

[0027] In one possible implementation, the identification module is used to identify, as the judgment result, that the occupant's body part has been projected through the window when the second detection result indicates that the occupant's body part has been projected through the window.

[0028] In one possible implementation mode, the step of recognizing a relative state between a body part of a passenger and a window in the image data and obtaining a first detection result includes: The method includes recognizing a relative state between a body part of a vehicle occupant and a window based on the image data, and obtaining a degree of confidence that the body part of the vehicle occupant has been pushed out of the window.

[0029] In one possible implementation, the specific module: If the reliability is greater than a predetermined threshold and the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; If the reliability is smaller than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; The reliability is used for at least one of the following: when the reliability is smaller than a predetermined threshold value; when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window; and when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window, the judgment result is used for at least one of the following: when the reliability is smaller than a predetermined threshold value;

[0030] In one possible implementation, the vehicle-mounted camera includes a camera installed inside the vehicle cabin and / or a camera installed outside the vehicle cabin.

[0031] In one possible implementation, the radar sensors are provided at the top / bottom edges of the window.

[0032] In one possible implementation, the sensing data further includes infrared sensing data and / or millimeter wave sensing data.

[0033] According to one aspect of the present disclosure, there is provided a vehicle monitoring system including at least two types of sensors for collecting at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in a window area; The present invention provides a vehicle including a controller that acquires the at least two types of sensing data from the sensor, detects whether or not a body part of a passenger in the vehicle has been exposed through a window based on the at least two types of sensing data, and generates warning information when a body part of the passenger has been exposed through the window.

[0034] According to one aspect of the present disclosure, there is provided an electronic device including a processor and a memory for storing instructions executable by the processor, the processor configured to invoke the instructions stored in the memory to perform the method for detecting when a body part of a vehicle occupant has been placed out of a window as described above.

[0035] According to one aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method for detecting when a body part of a vehicle occupant has been placed out of a window as described above.

[0036] According to one aspect of the present disclosure, there is provided a computer program product including computer readable code or a computer readable storage medium having computer readable code stored thereon, the computer readable code, when executed in a processor of an electronic device, causing the processor of the electronic device to perform the method for detecting when a body part of a vehicle occupant has been placed out of a window as described above.

[0037] In an embodiment of the present disclosure, whether a body part of a vehicle occupant has stuck out a window is detected based on at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting obstacles in the window area, and at least two types of detection results are obtained. A determination result is identified based on the at least two types of detection results, and if the determination result indicates that a body part of the occupant has stuck out a window, warning information is generated. According to the embodiment of the present disclosure, the determination result is identified by combining the image data and the radar sensing data, and warning information is generated, which is advantageous in improving the accuracy of the determination result and improving riding safety.

[0038] It should be understood that the foregoing general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0039] The drawings included as part of the specification illustrate embodiments of the present disclosure and, together with the specification, explain the technical solution of the present disclosure. [Figure 1] 1 is a flowchart illustrating a method for detecting a body part of a vehicle occupant sticking out of a window according to an embodiment of the present disclosure. [Figure 2] 1 is a schematic diagram illustrating a method for detecting a body part of a vehicle occupant sticking out of a window according to an embodiment of the present disclosure. [Figure 3] 1 is a block diagram illustrating an apparatus for detecting when a body part of a vehicle occupant has been placed out of a window according to an embodiment of the present disclosure. [Figure 4] FIG. 19 is a block diagram illustrating an electronic device 1900 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0040] Various exemplary embodiments, features, and aspects of the present disclosure are described in detail below with reference to the drawings, in which like reference numerals represent functionally identical or similar elements. While various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0041] The word "exemplary" herein means "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0042] In this specification, the term "and / or" merely describes a related relationship between related objects and indicates that three relationships can exist; for example, A and / or B can indicate three cases: A exists only, A and B exist simultaneously, and B exists only. Also, in this specification, the term "at least one" indicates any one of a plurality or any combination of at least two of a plurality; for example, including at least one of A, B, and C can indicate including any one or more elements selected from the set consisting of A, B, and C.

[0043] In addition, in order to better explain the present disclosure, many specific details are shown in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without some specific details. In some examples, methods, means, elements, and circuits that are well known to those skilled in the art are not described in detail in order to emphasize the gist of the present disclosure.

[0044] While the vehicle is moving, the driver or other passengers may place their hands, head, or other body parts out the window, which may result in a serious accident.

[0045] An embodiment of the present disclosure provides a technical solution for detecting whether a body part of a vehicle occupant has stuck out a window, and detects whether a body part of a vehicle occupant has stuck out a window based on at least two types of sensing data, obtains at least two systems of detection results, identifies a judgment result based on the at least two systems of detection results, and generates warning information when the judgment result indicates that the body part of the occupant has stuck out a window. According to the embodiment of the present disclosure, by combining image data and radar sensing data, it is possible to accurately identify whether a body part of the occupant has stuck out a window, and when the judgment result indicates that the body part of the occupant has stuck out a window, generates warning information, which is advantageous to improving the riding safety of occupants.

[0046] 1 is a flowchart illustrating a method for detecting a body part of a vehicle occupant sticking out a window according to an embodiment of the present disclosure. In one possible implementation, the method for detecting a body part of a vehicle occupant sticking out a window may be performed by an electronic device such as an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, terminal device such as a cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, or wearable device, or a server. The in-vehicle device may be an in-vehicle controller, domain controller, or in-vehicle infotainment device, or may be a device host for performing image or other data processing operations, such as an occupant monitoring system (OMS) or a driver monitoring system (DMS). In some possible implementations, the method for detecting a body part of a vehicle occupant sticking out a window may be implemented by a processor invoking computer-readable instructions stored in a memory. As shown in FIG. 1, the above-described method for detecting that a body part of a vehicle occupant has stuck out of a window includes steps S11 to S13.

[0047] In step S11, based on at least two types of sensing data including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting obstacles in the window area, it is detected whether or not a body part of a vehicle occupant has protruded out of the window, and at least two types of detection results are obtained.

[0048] In step S12, a determination result is determined based on the detection results of the at least two systems.

[0049] In step S13, when the determination result indicates that the body part of the occupant has protruded out of the window, attention-calling information is generated.

[0050] The embodiments of the present disclosure may be applied to any type of vehicle, such as a passenger car, a taxi, an online ride-hailing service, a shared car, or a bus. The vehicle type may be, for example, a compact car, a sport utility vehicle (SUV), or the like, but is not limited thereto in the embodiments of the present disclosure. The vehicle occupant may include any person riding in the vehicle. For example, the vehicle occupant may include a driver and / or a non-driver. For example, in a private car application, the non-driver may be the owner's family or friends. In an online ride-hailing application, the non-driver may be a passenger. The body part of the occupant may be a part of the occupant's body and may include at least one of a hand, an arm, a head, a foot, and a leg.

[0051] In one possible implementation, if the vehicle has two types of sensors for detecting whether a body part of a vehicle occupant has stuck out of a window, two types of sensing data can be acquired in step S11, for example, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in the window area.

[0052] Regarding image data collected by an onboard camera, the onboard camera may include a camera installed inside the vehicle and / or a camera installed outside the vehicle. The onboard camera may be any camera installed in the vehicle. The number of onboard cameras may be one or more. The type of onboard camera may be a DMS camera, an OMS camera, an exterior camera, etc., but this disclosure is not limited to these. The image data collected by the onboard camera may be image data of the vehicle environment captured by a camera installed inside or outside the vehicle, such as a DMS camera, an OMS camera, or an exterior camera. This image data must include image information of at least the seating areas and window areas of the vehicle, i.e., at least some seating areas of people and at least some window areas within the field of view of the camera.

[0053] It should be understood that with the widespread use of OMS systems and DMS systems in vehicles, if the on-board OMS cameras and DMS cameras can be used directly, it is possible to detect whether or not a passenger's body part has been placed out of the window without the need for an additional camera, which is advantageous for reducing costs.

[0054] Regarding the radar sensing data collected by the radar sensor, the radar sensor may be provided at the upper / lower edge of the window. For example, a radar sensor may be provided at the upper / lower edge of each window of the vehicle. The number of radar sensors may be one or more. The type of radar sensor may be an ultrasonic radar, an electromagnetic radar, a laser radar, etc., but is not limited thereto in the present disclosure. The radar sensing data collected by the radar sensor may be radar sensing data collected by a radar sensor such as an ultrasonic radar, an electromagnetic radar, or a laser radar provided at the upper / lower edge of each window, and is used to detect whether an obstacle exists in each window area.

[0055] After acquiring the image data and the radar sensing data, it is possible to detect whether or not a body part of a vehicle occupant has stuck out of the window based on each of these two types of data, thereby obtaining a first detection result based on the image data and a second detection result based on the radar sensing data.

[0056] In one possible implementation, a relative state between a body part of a passenger and a window in the image data is recognized to obtain a first detection result.

[0057] Based on the image data, a relative state between the vehicle occupant and the window in the image data is recognized, thereby detecting whether a body part of the vehicle occupant has protruded through the window, thereby obtaining a first detection result. The relative state indicates the relative position or overlapping relationship between the occupant's body and the window, and may include whether the occupant's body part is outside or inside the window. As an alternative, the image data may be processed using a neural network model to recognize the relative state between the vehicle occupant and the window in the image data. For example, a large amount (1,000 images) of image data related to the vehicle interior environment may be used to pre-train a neural network model for detecting whether a body part of the vehicle occupant has protruded through the window, and the image data may be input to the trained neural network model to obtain a first detection result. Here, the neural network model for detecting whether a body part of a vehicle occupant is sticking out of a window may be a neural network model such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a generative adversarial net (GAN), but the present disclosure is not limited to this. As an optional form, the relative state of the vehicle occupant and the window may be identified by detecting the window and the body part of the person in the image data, determining whether their positions overlap based on the detection results, and determining whether at least a part of the body part of the person is outside the vehicle body.

[0058] In one possible implementation, a second detection result is obtained based on the radar sensing data.

[0059] Based on the radar sensing data, it may be detected whether a body part of a vehicle occupant has protruded from a window, thereby obtaining a second detection result. The radar sensor is for detecting obstacles and may be located, for example, outside the window, below the lower edge of the window, or at least one of the upper and lower edges of the window. Whether an obstacle exists between the upper and lower edges of the window may be determined based on the radar sensing data, or the distance to the detected obstacle (e.g., the distance between the obstacle and the radar sensor) may be acquired based on the radar sensing data, and the second detection result may be obtained by determining whether the distance to the obstacle is shorter than a preset distance (e.g., the distance to the obstacle detected by the radar sensor located at the lower edge of the window is shorter than the distance between the upper edge of the window and the radar sensor). Here, multiple radar sensors may be provided on the upper / lower edges of the same window. The second detection result may be obtained by comprehensively analyzing multiple radar sensing data collected by multiple radar sensors on the window. However, the present disclosure does not limit the specific method for identifying the second detection result based on the radar sensing data.

[0060] In step S11, a first detection result based on image data and a second detection result based on radar sensing data are obtained. In step S12, a determination result can be determined based on the first detection result and the second detection result. Here, the determination result can be determined using various determination methods, but the present disclosure does not limit the specific determination method.

[0061] For example, assume that the values ​​of the first and second detection results corresponding to the same window can be represented by "0" and "1," where "0" indicates that the body part of the vehicle occupant is not sticking out of the window, and "1" indicates that the body part of the vehicle occupant is sticking out of the window.

[0062] In one possible implementation, an OR operation is performed on the first detection result and the second detection result, and the result of the OR operation of the first detection result and the second detection result can be specified as the determination result. In this way, if either the first detection result or the second detection result indicates that a body part of a vehicle occupant has stuck out through the window, a determination result that the body part of a vehicle occupant has stuck out through the window can be obtained, which is advantageous in reducing the rate of missing a determination that a body part of a vehicle occupant has stuck out through the window.

[0063] Alternatively, in another possible implementation form, an AND operation may be performed on the first detection result and the second detection result, and the result of the AND operation on the first detection result and the second detection result may be determined as the determination result. In this way, only when both the first detection result and the second detection result indicate that a body part of the occupant has been stuck out of the window, is the determination result determined to be that the body part of the occupant has been stuck out of the window, which is advantageous in reducing the rate of misidentification of a body part of a vehicle occupant having been stuck out of the window.

[0064] Alternatively, in another possible implementation, preset weights may be set for the first detection result and the second detection result, and the magnitude of the influence of the first detection result and the second detection result on the judgment result may be determined based on the magnitude of the preset weight, thereby obtaining the final judgment result.

[0065] In another possible implementation, radar sensing data may be used as a reference. If the second detection result based on the radar sensing data indicates that the body part of the occupant has been projected out of the window, the determination result is identified as having the body part of the occupant projected out of the window. In one possible implementation, determining the determination result based on the first detection result and the second detection result includes at least one of: when both the first detection result and the second detection result indicate that the body part has been projected out of the window, the determination result is identified as having the body part of the occupant projected out of the window; when the first detection result indicates that the body part has been projected out of the window and the second detection result indicates that the body part has not been projected out of the window, the determination result is identified as having the body part of the occupant not projected out of the window; and when the first detection result indicates that the body part has not been projected out of the window and the second detection result indicates that the body part has been projected out of the window, the determination result is identified as having the body part of the occupant projected out of the window.

[0066] Here, in the present disclosure, the magnitude of the preset weights is not limited. By setting the preset weights for the first detection result and the second detection result, an appropriate determination method can be selected for a specific scene, which is advantageous in improving the accuracy of the determination result.

[0067] In step S12, the determination result is identified. In step S13, if the determination result indicates that a body part of the occupant has protruded out of the window, attention-calling information can be generated.

[0068] For example, if the determination result indicates that the occupant has stuck a body part out of the window, warning information such as "Do not stick your body out of the window" can be displayed via the in-vehicle infotainment and / or the center console, etc. Here, the warning information may be in the form of text warning information, audio warning information, screen warning information, etc., but the present disclosure is not limited thereto.

[0069] In one possible implementation, the vehicle may have more than two types of sensors for detecting whether a body part of a vehicle occupant has been placed out of the window, i.e., in addition to an on-board camera and a radar sensor, the vehicle may also be provided with other sensors for detecting obstacles in the window area, such as infrared sensors, millimeter wave sensors, etc.

[0070] After identifying the first and second detection results based on the image data and radar sensing data, it may be possible to detect whether a body part of a vehicle occupant has stuck out a window based on each of sensing data collected by other types of sensors (e.g., infrared sensing data and / or millimeter-wave sensing data), and identify corresponding detection results of other systems. A determination result may be identified from the detection results of multiple systems, and if the determination result indicates that a body part of the occupant has stuck out a window, warning information may be generated. Here, the more types of sensing data acquired, the more detection results are obtained from each type of sensor, and the more accurate the determination result based on the detection results of the more systems. For specific processes, please refer to the above description regarding the two types of sensors, and detailed description will be omitted here.

[0071] Therefore, in an embodiment of the present disclosure, it is detected whether a body part of a vehicle occupant has stuck out a window based on each of at least two types of sensing data, at least two systems of detection results are obtained, a determination result is identified based on the at least two systems of detection results, and if the determination result indicates that a body part of the occupant has stuck out a window, warning information is generated. According to the embodiment of the present disclosure, it is possible to accurately identify whether a body part of the occupant has stuck out a window, and if the determination result indicates that a body part of the occupant has stuck out a window, warning information can be generated, which is advantageous to improving the safety of occupants in the vehicle cabin.

[0072] A method for detecting when a body part of a vehicle occupant has been placed out of a window according to an embodiment of the present disclosure will now be described.

[0073] 2 is a schematic diagram illustrating a method for detecting whether a body part of a vehicle occupant has stuck out a window according to an embodiment of the present disclosure. As shown in FIG. 2, data for detecting whether a body part of a vehicle occupant has stuck out a window may include radar sensing data collected by a radar sensor, image data collected by an on-board camera, and vehicle body signal data. By comprehensively analyzing and processing the radar sensing data, image data, and vehicle body signal data by a processor, it is possible to determine whether a body part of the occupant has stuck out a window, and to generate warning information when a body part of the occupant has stuck out a window.

[0074] It should be understood that the embodiment shown in FIG. 2 is merely exemplary, and the sensing data according to the embodiments of the present disclosure may further include radar sensing data collected by a radar sensor for detecting obstacles in a window area, millimeter wave sensing data collected by a millimeter wave sensor for detecting obstacles in a window area, etc., but the present disclosure is not limited thereto.

[0075] In one possible implementation, the method further includes obtaining vehicle body signal data including door data indicating whether a door is open or closed, and window data indicating whether a window is open or closed.

[0076] For example, vehicle body signal data can be acquired from a vehicle body bus such as a Local Interconnect Network (LIN) bus or a Controller Area Network (CAN) bus, and the open / closed status of each door and each window can be acquired based on the vehicle body signal data.

[0077] Here, in addition to acquiring vehicle body signal data from the vehicle body bus, vehicle body signal data indicating the open / closed state of the doors and windows can be acquired by installing sensors in the door area or window area, but the present disclosure does not limit the specific method for acquiring vehicle body signal data.

[0078] In addition, step S11 may include, when the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the passenger is open, detecting whether or not a body part of the passenger has protruded out of the window based on each of at least two types of sensing data of the vehicle, and obtaining the at least two systems of detection results.

[0079] In one possible implementation, vehicle body signal data and at least two kinds of sensing data (e.g., including image data and radar sensing data) can be acquired in real time. When the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of a passenger is open, a first detection result can be obtained by detecting whether a body part of the passenger corresponding to the open window in the image data has protruded through the window, and a second detection result can be obtained by detecting whether a body part of the passenger has protruded through the window, based on the image data, and a second detection result can be obtained by detecting whether a body part of the passenger has protruded through the window, based on the radar sensing data.

[0080] For example, suppose the vehicle body signal data indicates that the door is closed and the window corresponding to the driver's seat position is open, in this case, based on the image data, a region of interest, i.e., a region including the driver and the driver's seat window, is extracted from the image data, and whether or not the driver's body part is sticking out from the window in the region is detected to obtain a first detection result, and based on the radar sensing data corresponding to the driver's seat window, whether or not the driver's body part is sticking out from the window is detected to obtain a second detection result.

[0081] In this way, vehicle body signal data and at least two types of sensing data are acquired in real time, and when the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of a passenger is open, detection is performed on each of the at least two types of sensing data, thereby obtaining at least two detection results. This method is simple and easy to implement, which is advantageous in improving the efficiency of on-site implementation.

[0082] In one possible implementation, if the vehicle body signal data indicates that the door is closed and the window corresponding to the passenger's seating position is open, in step S11, a trigger signal is sent to the radar sensor to collect sensing data, and radar sensing data collected by the radar sensor is acquired; image data is acquired by the onboard camera; and based on the radar sensing data and the image data, it is detected whether or not a body part of the passenger has protruded from the window corresponding to the passenger's seating position, and a detection result based on the radar sensor and a detection result based on the image data are obtained.

[0083] For example, if vehicle body signal data is previously acquired and the vehicle body signal data indicates that a door is closed and a window corresponding to the passenger's seating position is open, a radar sensor is triggered to collect radar sensing data and image data collected by an on-board camera is acquired. For example, if the vehicle body signal data indicates that a door is closed and a window corresponding to the passenger's seating position is open, a trigger signal (e.g., one or more high pulse signals) can be sent to the radar sensor, causing the radar sensor to activate for detecting obstacles in the window area and collect radar sensing data. It should be understood that the on-board camera is in an active state because it can continuously acquire image data. If the vehicle body signal data indicates that a door is closed and a window corresponding to the passenger's seating position is open, image data collected by the on-board camera can be obtained without sending a signal to trigger its operation.

[0084] Based on the image data, it may be detected whether a body part of the occupant corresponding to an open window in the image data has protruded through the window, thereby obtaining a first detection result, and based on the radar sensing data, it may be detected whether the body part of the occupant has protruded through the window, thereby obtaining a second detection result.

[0085] In this way, when the trigger condition is met, the radar sensor is triggered to collect radar sensing data, which is advantageous for reducing energy consumption and extending the useful life of the radar sensor.

[0086] In one possible implementation, if the vehicle body signal data indicates that a door is open or that all windows corresponding to the passenger's seating position are closed, the determination result is that the passenger's body part is not sticking out of the window.

[0087] For example, when the vehicle door is open, the vehicle may be in a non-moving state, e.g., passengers getting in or out. While the vehicle is in operation, there is a significant hidden danger behind the act of a vehicle passenger sticking their body parts out of the window. Conversely, for a vehicle that is not in a moving state, it is not very meaningful to detect whether a passenger's body parts are sticking out of the window. Therefore, if the vehicle body signal data indicates that the door is open (meaning that the vehicle may be in a non-moving state) or that the windows corresponding to the passenger's seating positions are both closed, the determination result can directly determine that the passenger's body parts are not sticking out of the window, without taking into account the image data and radar sensing data.

[0088] In this way, it is possible to directly obtain a determination result that the passenger's body part is not sticking out of the window without needing to detect image data and radar sensing data, thereby improving the efficiency of the determination.

[0089] In another possible scenario, image data may be acquired first, and then radar sensing data (or radar sensing data and vehicle body signal data) corresponding to a target window in the image data may be acquired according to the image data, the target window being a target window corresponding to a target occupant in the image data who has a body part sticking out of the window.

[0090] In one possible implementation, step S11 comprises: Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; When the first detection result indicates that a body part has been stuck out of a window, acquiring seat information of a target passenger who has stuck out a body part in the image data through a window based on the first detection result; Obtaining the radar sensing data of a target window corresponding to seat information of the target passenger; and identifying a second detection result based on the radar sensing data of the target window.

[0091] For example, the image data may include multiple passengers and multiple windows, and the relative state of each passenger's body part relative to the window in the image data can be recognized, and it can be detected whether or not each passenger's body part is sticking out of the window, thereby obtaining a first detection result. For example, suppose that certain image data includes passengers A to C, and passenger B's body part is sticking out of the window. By inputting this image data into a trained neural network, it is possible to obtain a first detection result for this image data, i.e., that the body parts of passengers A and C are not sticking out of the window, and that passenger B's body part is sticking out of the window.

[0092] If the first detection result indicates that a body part of the occupant has stuck out the window, occupant B who has stuck out the body part of the window according to the first detection result is determined as the target occupant, and seat information related to the seat in which this occupant is sitting (for example, the passenger seat position) is identified. The window closest to the seat information of target occupant B is determined as the target window, for example, the passenger seat window corresponding to the passenger seat position. Radar sensing data of the passenger seat window is acquired, and a second detection result is identified based on this radar sensing data.

[0093] In this way, radar sensing data of the target window is acquired according to the image data, and the second detection result is determined based on the radar sensing data of the target window, which is advantageous in improving the accuracy of the second detection result.

[0094] In one possible implementation form, acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant includes acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant, and vehicle body signal data related to the seat information of the target occupant, including first signal data indicating the open / closed state of a target door, which is the door corresponding to the seat information of the target occupant, and second signal data indicating the open / closed state of the target window, and identifying the second detection result based on the radar sensing data includes identifying the second detection result based on the radar sensing data and the vehicle body signal data.

[0095] For example, in the first detection result, the body parts of passengers A and C are not sticking out the window, but the body part of passenger B, who is sitting in the passenger seat, is sticking out the window. Here, passenger B, who has stuck his body part out the window, is the target passenger. From the first detection result, it is possible to identify that the seat information of target passenger B is the passenger seat position. Radar sensing data of the passenger seat window (the window adjacent to the passenger seat) and vehicle body signal data related to the passenger seat position are acquired. Here, the open / closed states of the passenger seat door and passenger seat window can be determined based on the vehicle body signal data related to the passenger seat position, and the vehicle body signal data related to the passenger seat position may include first signal data indicating the open / closed state of the door at the passenger seat position and second signal data indicating the open / closed state of the window at the passenger seat position.

[0096] A second detection result can be determined based on the radar sensing data of the passenger door, the first signal data, and the second signal data, where if the first signal data indicates that the passenger door is open or the second signal data indicates that the passenger window is closed, the second detection result can be determined to be that the body part of the passenger in the passenger seat is not sticking out of the passenger window.

[0097] It should be understood that the above process is merely illustrative, and in an embodiment of the present disclosure, radar sensing data and associated vehicle body signal data corresponding to the seat information of each target occupant in the image data can be obtained, and a second detection result corresponding to each target occupant can be identified based on the radar sensing data and associated vehicle body signal data corresponding to the seat information of each target occupant, respectively.

[0098] In this way, radar sensing data corresponding to the seat information of the target occupant and vehicle body signal data related to the seat information of the target occupant can be obtained according to the image data, and the second detection result can be determined based on the radar sensing data of the target vehicle window and its related vehicle body signal data, which is advantageous for further improving the accuracy of the second detection result.

[0099] In the above-described example of step S11, when the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the occupant is open, only when the first detection result indicates that a body part of the occupant has been stuck out of the window, corresponding radar sensing data, or radar sensing data and vehicle body signal data, are acquired to identify a second detection result. Thus, in the specific example of step S12, when the second detection result indicates that a body part of the occupant has been stuck out of the window, the determination result is that the body part of the occupant has been stuck out of the window.

[0100] For example, assuming that the target occupant whose body part is sticking out the window in the image data is a passenger seat occupant, if the second detection result indicates that the passenger seat occupant's body part is sticking out the window, the determination result is that the passenger seat occupant's body part is identified as sticking out the window. The seat of the target occupant may be any seat, but it should be understood that this disclosure is not limited thereto.

[0101] This is advantageous in improving the accuracy of the determination results.

[0102] In one possible implementation form, the step of recognizing the relative state of the vehicle occupant's body part and the window in the image data and obtaining a first detection result includes recognizing the relative state of the vehicle occupant's body part and the window based on the image data and obtaining a confidence level that the vehicle occupant's body part has been projected out the window.

[0103] For example, recognizing the relative state of the body part of the vehicle occupant and the window based on the image data may include inputting the image data to a trained neural network model, recognizing the relative state of the body part of the vehicle occupant and the window using the neural network model, and obtaining a first detection result, which may include a confidence level that the body part of the occupant has been placed out the window.

[0104] Here, the larger the reliability value, the higher the probability that the body part has been put out through the window, and the smaller the reliability value, the lower the probability that the body part has been put out through the window.

[0105] In this way, it is possible to obtain a degree of reliability as to whether a body part of a vehicle occupant has been exposed through the window, which is advantageous in improving the accuracy of the determination result indicating whether a body part of a vehicle occupant has been exposed through the window.

[0106] In step S11, the reliability of the first detection result is determined. In step S12, the determination result can be determined based on the reliability of the first detection result and the second detection result.

[0107] In one possible implementation, in step S12: If the reliability is greater than a predetermined threshold and the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; If the reliability is greater than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant is not sticking out of a window, identifying as the determination result that a body part of the vehicle occupant is not sticking out of a window; If the reliability is smaller than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; The method includes at least one of: when the reliability is smaller than a predetermined threshold value; when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window; and when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window, identifying the body part of the vehicle occupant as not sticking out of the window as the judgment result.

[0108] For example, if the reliability is greater than a preset threshold (e.g., 0.5), it is determined that the reliability of the body part of the vehicle occupant sticking out of the window is high based on the image data. In this case, if the second detection result indicates that the body part of the vehicle occupant sticks out of the window, it is determined as the determination result that the body part of the vehicle occupant sticks out of the window.

[0109] If the reliability is smaller than a preset threshold (e.g., 0.5), it is determined that the reliability of the occupant's body part sticking out the window is low based on the image data. In this case, the determination result can be determined based on the second detection result. Specifically, if the second detection result indicates that the vehicle occupant's body part has stuck out the window, it is determined that the vehicle occupant's body part has stuck out the window, and if the second detection result indicates that the vehicle occupant's body part has not stuck out the window, it is determined that the vehicle occupant's body part has not stuck out the window.

[0110] In this way, the reliability and the second detection result can be taken into consideration comprehensively, which is advantageous in improving the accuracy of the determination result.

[0111] The above describes several methods for determining the judgment result based on vehicle body signal data and at least two types of sensing data, but it should be understood that the method for determining the judgment result can be flexibly selected depending on the actual situation.

[0112] The determination result is specified in steps S11 and S12. In step S13, if the determination result indicates that a body part of the occupant has protruded out of the window, attention-calling information can be generated.

[0113] Therefore, in an embodiment of the present disclosure, it is detected whether a body part of a vehicle occupant has stuck out a window based on each of at least two types of sensing data, at least two systems of detection results are obtained, a determination result is identified based on the at least two systems of detection results, and if the determination result indicates that a body part of the occupant has stuck out a window, attention alert information is generated. According to the embodiment of the present disclosure, by combining image data and radar sensing data, it is possible to accurately identify whether a body part of the occupant has stuck out a window, and if the determination result indicates that a body part of the occupant has stuck out a window, attention alert information can be generated, which is advantageous to improving the safety of occupants in the vehicle cabin.

[0114] It should be understood that the above method embodiments mentioned in this disclosure can be combined with each other to form embodiments without violating the principles and logic. Due to space limitations, detailed descriptions are omitted in this disclosure. Those skilled in the art will understand that in the above methods according to specific embodiments, the execution order of each step depends on its function and possible underlying logic.

[0115] In addition, the present disclosure further provides a device, an electronic device, a computer-readable storage medium, and a program for detecting when a body part of a vehicle occupant has been pushed out of a window, all of which can be used to realize any of the methods for detecting when a body part of a vehicle occupant has been pushed out of a window according to the present disclosure. For such technical solutions and descriptions, please refer to the descriptions in the description of the method, and detailed descriptions will be omitted here.

[0116] 3 is a block diagram illustrating an apparatus for detecting a body part of a vehicle occupant sticking out of a window according to an embodiment of the present disclosure. As shown in FIG. 3, the apparatus includes: a detection module 71 for detecting whether a body part of a vehicle occupant has stuck out a window based on at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting obstacles in the window area, and for obtaining at least two types of detection results; an identification module 72 for identifying a judgment result based on the at least two detection results; and an attention module 73 for generating attention information when the determination result indicates that the body part of the occupant has protruded out of the window.

[0117] In one possible implementation, the device comprises: further comprising an acquisition module for acquiring vehicle body signal data including door data indicating an open / closed state of a door and window data indicating an open / closed state of a window; When the vehicle body signal data indicates that the door is closed and the window corresponding to the passenger's seating position is open, the detection module 71 is used to detect whether the passenger's body part has protruded out of the window based on each of at least two types of sensing data from the vehicle, and to obtain the at least two systems of detection results.

[0118] In one possible implementation, if the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of a passenger is open, the detection module 71: Transmitting a trigger signal to the radar sensor to collect sensing data, and acquiring the radar sensing data collected by the radar sensor; acquiring image data collected by the on-board camera; It is used to detect whether or not a body part of the occupant has protruded from a window corresponding to the occupant's seating position based on the radar sensing data and the image data, respectively, and to obtain detection results based on the radar sensor and detection results based on the image data.

[0119] In one possible implementation, the device is further used to determine, as the determination result, that the occupant's body part is not sticking out of the window when the vehicle body signal data indicates that a door is open or that all windows corresponding to the occupant's seating position are closed.

[0120] In one possible implementation, the detection module 71 recognizes a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; and obtaining a second detection result based on the radar sensing data; The determination module 72 is used to determine a judgment result based on the first detection result and the second detection result.

[0121] In one possible implementation, the identification module 72: When both the first detection result and the second detection result indicate that a body part of the occupant has been stuck out of a window, specifying as the determination result that the body part of the occupant has been stuck out of a window; When the first detection result indicates that a body part of the passenger has been put out of a window and the second detection result indicates that the body part has not been put out of a window, determining that the body part of the passenger has not been put out of a window as the determination result; When the first detection result indicates that a body part has not been put out through the window and the second detection result indicates that a body part has been put out through the window, the determination result is used to at least one of identify that the body part of the occupant has been put out through the window.

[0122] In one possible implementation, the detection module 71 Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; When the first detection result indicates that a body part has been stuck out of a window, acquiring seat information of a target passenger who has stuck out a body part in the image data through a window based on the first detection result; Obtaining the radar sensing data of a target window corresponding to seat information of the target passenger; and determining a second detection result based on the radar sensing data of the target window.

[0123] In one possible implementation form, the acquiring of the radar sensing data of the target window corresponding to the seat information of the target occupant includes: acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant, and vehicle body signal data related to the seat information of the target occupant, including first signal data indicating an open / close state of a target door that is a door corresponding to the seat information of the target occupant, and second signal data indicating an open / close state of the target window; The step of identifying the second detection result based on the radar sensing data of the target window includes: The method includes identifying the second detection result based on the radar sensing data and the vehicle body signal data.

[0124] In one possible implementation, when the second detection result indicates that the occupant's body part has been projected out of the window, the identification module 72 is used to identify the occupant's body part as having been projected out of the window as the judgment result.

[0125] In one possible implementation mode, the step of recognizing a relative state between a body part of a passenger and a window in the image data and obtaining a first detection result includes: The method includes recognizing a relative state between a body part of a vehicle occupant and a window based on the image data, and obtaining a degree of confidence that the body part of the vehicle occupant has been pushed out of the window.

[0126] In one possible implementation, the identification module 72 is configured to, when the confidence level is greater than a preset threshold, identify the body part of the vehicle occupant as having been stuck out of a window when the second detection result indicates that the body part of the vehicle occupant has been stuck out of a window as the determination result; If the reliability is smaller than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; The reliability is used for at least one of the following: when the reliability is smaller than a predetermined threshold value; when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window; and when the second detection result indicates that the body part of the vehicle occupant is not sticking out of the window, the judgment result is used for at least one of the following: when the reliability is smaller than a predetermined threshold value;

[0127] In one possible implementation, the vehicle-mounted camera includes a camera installed inside the vehicle cabin and / or a camera installed outside the vehicle cabin.

[0128] In one possible implementation, the radar sensors are provided at the top / bottom edges of the window.

[0129] In one possible implementation, the sensing data further includes infrared sensing data and / or millimeter wave sensing data.

[0130] In some embodiments, the functions or modules of the apparatus according to the embodiments of the present disclosure are used to perform the methods described in the above method embodiments, and the specific implementations will be apparent from the description of the above method embodiments, and detailed descriptions will be omitted for the sake of brevity.

[0131] The embodiment of the present disclosure further includes at least two types of sensors for collecting at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in a window area; The present invention provides a vehicle including a controller that acquires the at least two types of sensing data from the sensor, detects whether or not a body part of a passenger in the vehicle has been exposed through a window based on the at least two types of sensing data, and generates warning information when a body part of the passenger has been exposed through the window.

[0132] Here, the controller may be used to implement the method for detecting whether a body part of a vehicle occupant has been pushed out of a window as described in the above embodiment. The sensing data may include other types of sensing data, such as infrared sensing data and / or millimeter wave sensing data. Exemplary implementations of the controller and the sensor may be referred to above, and detailed descriptions thereof will be omitted.

[0133] An embodiment of the present disclosure further provides a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, causing the computer-readable storage medium to implement the method for detecting when a body part of a vehicle occupant has been placed out of a window, as described above. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0134] An embodiment of the present disclosure further provides an electronic device including a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to invoke the instructions stored in the memory to perform the method for detecting when a body part of a vehicle occupant has been placed out of a window as described above.

[0135] An embodiment of the present disclosure further provides a computer program product including computer readable code or a computer readable storage medium having computer readable code stored thereon, the computer readable code, when executed in a processor of an electronic device, causing the processor of the electronic device to perform the above-described method for detecting when a body part of a vehicle occupant has been placed out of a window.

[0136] The electronic device may be provided as a terminal, a server, or some other form of device.

[0137] FIG. 4 is a block diagram illustrating an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 4, the electronic device 1900 further includes a processing component 1922 including one or more processors, and memory resources, represented by memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. The processing component 1922 is also configured to perform the above-described method by executing the instructions.

[0138] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as the Microsoft Windows Server operating system (Windows Server™), the Apple graphical user interface operating system Mac OS X™, a multitasking, multiuser operating system for computers (Unix™), a free and open source Unix-based operating system (Linux™), an open source Unix-based operating system (FreeBSD™), or the like.

[0139] In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 1932, is further provided that includes computer program instructions that, when executed by the processing component 1922 of the electronic device 1900, can cause the above-described methods to be performed.

[0140] The present disclosure may be a system, a method, and / or a computer program product, which may include a computer-readable storage medium having computer-readable program instructions for causing a processor to implement aspects of the present disclosure.

[0141] A computer-readable storage medium may be a tangible device capable of storing and storing instructions for use in an instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Further specific examples (non-exhaustive list) of computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or slot-in protrusion structures on which instructions are stored, and any suitable combination of the above. As used herein, computer-readable storage media is not to be construed as a transitory signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated via a waveguide or other transmission medium (e.g., pulsed light passing through a fiber optic cable), or electrical signals transmitted via electrical wires.

[0142] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium into each computing / processing device, or may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0143] Computer program instructions for carrying out the operations of the present disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or target code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and general procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, the state information of the computer-readable program instructions may be used to personalize electronic circuitry, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), to execute the computer-readable program instructions to implement aspects of the present disclosure.

[0144] Although various aspects of the present disclosure have been described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure, it should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0145] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus such that, when executed by the processor of the computer or other programmable data processing apparatus, the instructions cause the apparatus to perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium to cause a computer, programmable data processing apparatus, and / or other device to operate in a determined manner. Thus, a computer-readable storage medium having instructions stored thereon includes an article of manufacture having instructions thereon that implements each aspect of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0146] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps, thereby creating a computer-implementable process. In this manner, the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0147] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of instructions, which may include one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions depicted in the blocks may be performed in a different order than that depicted in the drawings. For example, two consecutive blocks may be executed substantially in parallel, or may be executed in a reverse order depending on the functionality. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0148] The computer program product may be specifically implemented by hardware, software, or a combination thereof. In one possible embodiment, the computer program product may be specifically implemented as a computer storage medium. In another possible embodiment, the computer program product may be specifically implemented as a software product, such as a software development kit (SDK).

[0149] Although the embodiments of the present disclosure have been described above, the above description is merely illustrative and not exhaustive, and is not limited to the disclosed embodiments. Various modifications and alterations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms chosen in this specification are intended to appropriately interpret the principles, practical applications, or technical improvements of the embodiments, or to allow other skilled in the art to understand the embodiments disclosed herein.

Claims

1. Acquiring vehicle body signal data including door data indicating the open / closed state of a door and window data indicating the open / closed state of a window; When the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the vehicle occupant is open, detecting whether or not a body part of the vehicle occupant has protruded from the window based on at least two types of sensing data including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting obstacles in the window area, and obtaining at least two systems of detection results; determining a determination result based on the at least two detection results; generating warning information when the determination result indicates that the body part of the occupant has been stuck out of the window.

2. When the vehicle body signal data indicates that the door is closed and the window corresponding to the seating position of the vehicle occupant is open, detecting whether or not a body part of the vehicle occupant has protruded from the window based on each of at least two types of sensing data, and obtaining at least two systems of detection results, Transmitting a trigger signal to the radar sensor to collect sensing data, and acquiring the radar sensing data collected by the radar sensor; acquiring image data collected by the on-board camera; 2. The method for detecting whether a body part of a vehicle occupant has protruded out of a window as described in claim 1, comprising: detecting whether a body part of the occupant has protruded out of a window corresponding to a seating position of the occupant based on the radar sensing data and the image data, respectively, and obtaining a detection result based on the radar sensor and a detection result based on the image data.

3. 2. The method for detecting whether a body part of a vehicle occupant has been exposed through a window as described in claim 1, further comprising: when the vehicle body signal data indicates that a door is open or that all windows corresponding to the occupant's seating position are closed, determining as the determination result that the occupant's body part has not been exposed through a window.

4. The above-mentioned detecting whether or not a body part of a vehicle occupant has stuck out a window based on each of at least two types of sensing data, and obtaining at least two types of detection results, Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; obtaining a second detection result based on the radar sensing data; The step of determining the determination result based on the at least two detection results includes: determining a determination result based on the first detection result and the second detection result; 2. The method for detecting when a body part of a vehicle occupant has been placed out of a window according to claim 1.

5. The determination result is determined based on the first detection result and the second detection result. What is that? When both the first detection result and the second detection result indicate that a body part of the occupant has been stuck out of a window, specifying as the determination result that the body part of the occupant has been stuck out of a window; When the first detection result indicates that a body part of the passenger has been put out of a window and the second detection result indicates that the body part has not been put out of a window, specifying that the body part of the passenger has not been put out of a window as the determination result; and if the first detection result indicates that the body part has not been stuck out the window and the second detection result indicates that the body part has been stuck out the window, identifying the body part of the occupant as having been stuck out the window as the determination result.

6. The above-mentioned detecting whether or not a body part of a vehicle occupant has stuck out a window based on each of at least two types of sensing data, and obtaining at least two types of detection results, Recognizing a relative state between a body part of a passenger and a window in the image data to obtain a first detection result; When the first detection result indicates that a body part has been stuck out of a window, acquiring seat information of the target passenger who has stuck out a body part in the image data through the window based on the first detection result; Obtaining the radar sensing data of a target window corresponding to seat information of the target passenger; The method for detecting when a body part of a vehicle occupant has gone out of a window according to claim 1 , further comprising: identifying a second detection result based on radar sensing data of the target window.

7. The acquiring of the radar sensing data of the target window corresponding to the seat information of the target passenger includes: acquiring the radar sensing data of a target window corresponding to the seat information of the target occupant, and vehicle body signal data related to the seat information of the target occupant, including first signal data indicating an open / close state of a target door that is a door corresponding to the seat information of the target occupant, and second signal data indicating an open / close state of the target window; The step of identifying the second detection result based on the radar sensing data of the target window includes: The method for detecting whether a body part of a vehicle occupant has stuck out a window according to claim 6 , further comprising: determining the second detection result based on the radar sensing data and the vehicle body signal data.

8. The step of determining the determination result based on the at least two detection results includes:

7. The method for detecting that a body part of a vehicle occupant has been stuck out of a window, as described in claim 6, further comprising, when the second detection result indicates that the body part of the occupant has been stuck out of the window, identifying the body part of the occupant as having been stuck out of the window as the determination result.

9. The above-mentioned recognizing a relative state between a body part of a passenger and a window in the image data and obtaining a first detection result includes:

5. The method for detecting whether a body part of a vehicle occupant has gone out of a window as described in claim 4, further comprising recognizing a relative state between the body part of the vehicle occupant and the window based on the image data, and obtaining a confidence level that the body part of the vehicle occupant has gone out of the window.

10. The step of determining the determination result based on the at least two detection results includes: If the reliability is greater than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; If the reliability is smaller than a predetermined threshold, and if the second detection result indicates that a body part of the vehicle occupant has stuck out a window, identifying the body part of the vehicle occupant as having stuck out a window as the determination result; 10. The method for detecting whether a body part of a vehicle occupant has been stuck out a window as described in claim 9, comprising at least one of: when the reliability is smaller than a predetermined threshold value; and when the second detection result indicates that the body part of the vehicle occupant has not been stuck out a window, identifying as the determination result that the body part of the vehicle occupant has not been stuck out a window.

11. The vehicle-mounted camera includes a camera installed inside the vehicle cabin and / or a camera installed outside the vehicle cabin, and / or the radar sensor is located at the top and / or bottom edge of the window; and / or The method for detecting whether a body part of a vehicle occupant has gone out of a window according to claim 1 , wherein the sensing data further includes infrared sensing data and / or millimeter wave sensing data.

12. Acquiring vehicle body signal data including door data indicating an open / closed state of a door and window data indicating an open / closed state of a window; a detection module for detecting whether a body part of a vehicle occupant has protruded from a window based on at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in the window area, when the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of the occupant is open, and for obtaining at least two detection results; an identification module for identifying a determination result based on the at least two detection results; and an alert module for generating alert information when the determination result indicates that the occupant's body part has been stuck out of the window.

13. Acquiring vehicle body signal data including door data indicating a door open / close state and window data indicating a window open / close state; at least two types of sensors for collecting at least two types of sensing data, including image data collected by an on-board camera and radar sensing data collected by a radar sensor for detecting an obstacle in a window area, when the vehicle body signal data indicates that a door is closed and a window corresponding to a seating position of a passenger is open; a controller that acquires the at least two types of sensing data from the sensor, detects whether a body part of a vehicle occupant has stuck out a window based on the at least two types of sensing data, and generates warning information when a body part of the occupant has stuck out the window.

14. a processor; a memory for storing instructions executable by the processor; The processor is configured to call instructions stored in the memory to execute the method for detecting when a body part of a vehicle occupant has been placed out of a window as described in any one of claims 1 to 11.

15. A computer-readable storage medium having stored thereon computer program instructions, the computer-readable storage medium causing, when executed by a processor, the computer-readable storage medium to implement the method for detecting when a body part of a vehicle occupant has been placed out of a window as claimed in any one of claims 1 to 11.

16. A computer program comprising computer-readable code, the computer-readable code running on a processor of an electronic device causing the processor of the electronic device to execute the method for detecting when a body part of a vehicle occupant has been placed out of a window according to any one of claims 1 to 11.

Images