Preventing side door accidents using existing time-of-flight (TOF) sensors

Abstract

Systems and methods for preventing side door accidents using existing time of flight (TOF) sensors include receiving sensor data captured by a sensor system of a vehicle and mapping the sensor data to generate a depth map of a proximity area around the vehicle. Here, the depth map includes objects in the proximity area around the vehicle. The system and method also include determining whether an object in the proximity area is within a region of interest corresponding to the door, and when the object is within the region of interest of the door, generating a notification for a user of the vehicle that alerts the user that the door cannot be opened.

Description

Technical Field

[0001] This disclosure generally relates to systems and methods for preventing vehicle door accidents using existing time-of-flight (TOF) sensors of a vehicle. Specifically, when a user exits the vehicle, sufficient space is required near the user's door to prevent damage from objects close to the door (such as other vehicles, walls, pillars, etc.). To remedy this, the user may need to reposition the vehicle within the parking space or find a completely new parking spot. Background Technology

[0002] The information provided in this section is intended to provide a general overview of the background of this disclosure. To the extent described in this section, the work of the currently named inventors, and aspects of the description that may not conform to the prior art at the time of submission, are neither explicitly nor implicitly acknowledged as prior art relative to this disclosure.

[0003] While dedicated sensors and cameras for collision prevention are available, these require additional equipment and vehicle integration, limiting their use in high-end vehicles. Therefore, a solution for detecting objects on the side of a vehicle is needed that can be universally applied to all vehicles manufactured by a single company, while limiting the added weight. Furthermore, when parking, users could benefit from advance notification of the risk of damage to vehicle doors and / or advance notification of insufficient parking space, prompting them to choose different parking spaces. Summary of the Invention

[0004] One aspect of this disclosure provides a computer-implemented method for preventing vehicle door accidents using existing time-of-flight (TOF) sensors. When executed on data processing hardware, the method causes the hardware to perform operations including: receiving sensor data captured by the vehicle's sensor system and mapping the sensor data to generate a depth map of a proximity area around the vehicle, the depth map including objects in the proximity area around the vehicle. The operation also includes determining whether an object in the proximity area is within a region of interest corresponding to the vehicle door. When an object is within the region of interest of the vehicle door, the operation further includes generating a notification for a user of the vehicle, warning the user that the door cannot be opened.

[0005] Embodiments of this disclosure may include one or more of the following optional features. In some embodiments, operation further includes receiving vehicle data, including one or more of vehicle position, vehicle steering, or vehicle speed. In these embodiments, mapping sensor data to generate a depth map of the proximity area around the vehicle may include aligning the vehicle data with sensor data captured by the vehicle's sensor system.

[0006] In some examples, the operation also includes receiving seat data indicating that a seat associated with a vehicle door is occupied. In these examples, generating a notification warning the user that the door cannot be opened may include identifying a region of interest corresponding to the occupied seat. In some implementations, generating a notification to the vehicle user includes displaying the notification on a display communicating with data processing hardware. In these implementations, generating a notification to the vehicle user may further include locking the inaccessible door, generating a visual alarm, or generating an audible alarm, or one or more of these.

[0007] In some examples, the region of interest is configured during the configuration process. In some implementations, the size of the region of interest is adjustable. In some examples, the operation also includes detecting the vehicle's gear position and generating a notification for the vehicle's user when the gear position indicates that the vehicle is parked.

[0008] Another aspect of this disclosure provides a computer-implemented method for preventing vehicle door accidents using existing time-of-flight (TOF) sensors. When executed on data processing hardware, the method causes the data processing hardware to perform operations including: receiving sensor data captured by the vehicle's sensor system while the vehicle is performing a parking procedure; mapping the sensor data to generate a depth map of the proximity area around the vehicle, the depth map including objects in the proximity area around the vehicle; and determining whether the objects in the proximity area are within a region of interest corresponding to the vehicle door. When the object is within the region of interest of the vehicle door, these operations also include displaying an indication highlighting the region of interest on a display communicating with the data processing hardware.

[0009] This aspect may include one or more of the following optional features. In some embodiments, operation further includes receiving vehicle data, which includes one or more of vehicle position, vehicle steering, or vehicle speed. In these embodiments, mapping sensor data to generate a depth map of the proximity area around the vehicle may include aligning the vehicle data with sensor data captured by the vehicle's sensor system.

[0010] Another aspect of this disclosure provides a system for preventing vehicle door accidents using existing Time-of-Flight (TOF) sensors. The system includes data processing hardware and memory hardware communicating with the data processing hardware. The memory hardware stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations including receiving sensor data captured by the vehicle's sensor system and mapping the sensor data to generate a depth map of a proximity area around the vehicle, the depth map including objects in the proximity area around the vehicle. The operations also include determining whether an object in the proximity area is within a region of interest corresponding to a vehicle door. When an object is within the region of interest of the vehicle door, the operations further include generating a notification for a user of the vehicle, warning the user that the door cannot be opened.

[0011] This aspect may include one or more of the following optional features. In some embodiments, operation further includes receiving vehicle data, which includes one or more of vehicle position, vehicle steering, or vehicle speed. In these embodiments, mapping sensor data to generate a depth map of the proximity area around the vehicle may include aligning the vehicle data with sensor data captured by the vehicle's sensor system.

[0012] In some examples, the operation also includes receiving seat data indicating that a seat associated with a door is occupied. In these examples, generating a notification warning the user that the door cannot be opened may include identifying a region of interest corresponding to the occupied seat and displaying the notification on a display communicating with data processing hardware. In these implementations, generating a notification to the vehicle user may further include locking the inaccessible door, generating a visual alarm, or generating an audible alarm, or one or more of these. In some examples, the region of interest is configured during the configuration process.

[0013] Details of one or more embodiments of this disclosure are set forth in the accompanying drawings and the following description. Other aspects, features, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0014] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0015] Figure 1 This is a schematic diagram of an example system that uses existing time-of-flight (TOF) sensors to prevent door accidents.

[0016] Figure 2 yes Figure 1 A schematic diagram of an example component of the system.

[0017] Figure 3A and Figure 3B yes Figure 1 A schematic diagram of the system's region of interest configuration process.

[0018] Figure 4 Is execution Figure 1 Example vehicles of the system.

[0019] Figure 5 It shows that it comes from Figure 1 Example user interface for system warnings.

[0020] Figure 6 It shows that it comes from Figure 1 Example user interface for system warnings.

[0021] Figure 7This is a flowchart of an example operational setup for a method of preventing door accidents using existing TOF sensors.

[0022] Figure 8 This is a flowchart of an example operational setup for a method of preventing door accidents using existing TOF sensors.

[0023] In all the accompanying drawings, the corresponding reference numerals indicate the corresponding parts. Detailed Implementation

[0024] The example configuration will now be described more fully with reference to the accompanying drawings. The example configuration is provided so that this disclosure will be thorough and will fully communicate the scope of this disclosure to those skilled in the art. Specific details, such as examples of specific components, devices, and methods, are set forth to provide a thorough understanding of the configuration of this disclosure. It will be apparent to those skilled in the art that the specific details are not required, that the example configuration may be implemented in many different forms, and that the specific details and exemplary configuration should not be construed as limiting the scope of this disclosure.

[0025] The terminology used herein is for the purpose of describing specific exemplary configurations only and is not intended to be limiting. As used herein, the singular articles “a,” “an,” and “the” may also be intended to include plural forms unless the context clearly indicates otherwise. The terms “comprising,” “including,” “containing,” and “having” are inclusive, thus specifying the presence of features, steps, operations, elements, and / or components, but not excluding the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein should not be construed as requiring them to be performed in the specific order discussed or shown, unless specifically identified as such. Additional or alternative steps may be employed.

[0026] When an element or layer is referred to as “on another element or layer,” “joined to,” “connected to,” “attached to,” or “linked to” another element or layer, it may be directly on, joined to, connected to, attached to, or linked to the other element or layer, or there may be intermediate elements or layers present. Conversely, when an element is referred to as “directly on another element or layer,” “directly joined to,” “directly connected to,” “directly attached to,” or “directly linked to” another element or layer, there may be no intermediate elements or layers present. Other terms used to describe relationships between elements should be interpreted in a similar manner (e.g., “between” vs. “directly between,” “adjacent” vs. “directly adjacent,” etc.). As used herein, the term “and / or” includes any and all combinations of one or more of the related listed items.

[0027] The terms first, second, third, etc., may be used herein to describe various elements, components, regions, layers, and / or parts. These elements, components, regions, layers, and / or parts should not be limited by these terms. These terms are used only to distinguish individual elements, components, regions, layers, or parts. Terms such as “first,” “second,” and other numerical terms do not imply order or sequence unless the context clearly indicates otherwise. Therefore, the first element, component, region, layer, or part discussed below may be referred to as the second element, component, region, layer, or part without departing from the teachings of the example configuration.

[0028] In this application, including the following definitions, the term "module" may be replaced by the term "circuit". The term "module" may refer to or be a part of an application-specific integrated circuit (ASIC), or include ASICs; digital, analog, or mixed-signal analog / digital discrete circuits; digital, analog, or mixed-signal analog / digital integrated circuits; combinational logic circuits; field-programmable gate arrays (FPGAs); processors (shared, dedicated, or grouped) that execute code; memory (shared, dedicated, or grouped) that stores code executed by the processor; other suitable hardware components that provide the functions described above; or some or all of the above combinations, such as in a system-on-a-chip.

[0029] The term "code" as used above can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, and / or objects. The term "shared processor" includes a single processor that executes some or all of the code from multiple modules. The term "group processor" includes a processor, in conjunction with an additional processor, that executes some or all of the code from one or more modules. The term "shared memory" includes a single memory that stores some or all of the code from multiple modules. The term "group memory" includes memory, in conjunction with additional memory, that stores some or all of the code from one or more modules. The term "memory" can be a subset of the term "computer-readable medium." The term "computer-readable medium" does not include transient electrical and electromagnetic signals propagating through the medium and can therefore be considered tangible, non-transitory memory. Non-limiting examples of non-transitory memory include tangible computer-readable media, including non-volatile memory, magnetic memory, and optical memory.

[0030] The apparatus and methods described in this application may be implemented, in whole or in part, by one or more computer programs executed by one or more processors. The computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer program may also include and / or depend on stored data.

[0031] A software application (i.e., a software resource) can refer to computer software that enables a computing device to perform tasks. In some examples, a software application may be referred to as an "application," "app," or "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and game applications.

[0032] Non-transitory memory can be a physical device used for temporary or permanent storage of programs (e.g., instruction sequences) or data (e.g., program state information) for use by a computing device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used in firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change memory (PCM), and magnetic disks or magnetic tapes.

[0033] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0034] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These different implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system, the programmable system including at least one programmable processor, at least one input device, and at least one output device, the programmable processor being dedicated or general-purpose, coupled to receive data and instructions from and send data and instructions to the storage system.

[0035] The processes and logic flows described in this specification can be executed by one or more programmable processors, also known as data processing hardware, which execute one or more computer programs to perform functions by manipulating input data and generating output. These processes and logic flows can also be executed by special-purpose logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include or be operatively coupled to one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from or transfer data to, or both. However, a computer does not need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented or incorporated therein by dedicated logic circuitry.

[0036] To provide interaction with the user, one or more aspects of this disclosure can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touchscreen, and optional keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0037] refer to Figure 1In some embodiments, system 100 includes vehicle 10 and / or a remote system 60 communicating with vehicle 10 via network 40 (e.g., wired or wireless communication). Vehicle 10 and / or remote system 60 implement side door protection system 200, configured to detect when there is insufficient space around vehicle 10 to open vehicle door 20, and to notify the user of vehicle 10 when there is insufficient space to open door 20. In short, as described further in detail below, side door protection system 200 is configured to receive sensor data 202 and determine whether any object 404 is present. Figure 4 Within the proximity area 70 of vehicle 10, and when object 404 is within the region of interest 18 corresponding to the door 20 of vehicle 10, a notification 232 is generated, warning the user that the door 20 of vehicle 10 cannot be opened. Note that by detecting whether there is sufficient lateral space available to open the door 20 of vehicle 10 when the vehicle is parked, the side door protection system 200 can prevent damage to vehicle 10 and / or other objects near vehicle 10 (e.g., other vehicles, buildings, people, etc.). Furthermore, when vehicle 10 executes parking procedure 400... Figure 4 In the event of insufficient parking space, the side door protection system 200 can notify the user of vehicle 10 in advance that there is not enough parking space, which can prompt the user to choose a different space to park.

[0038] In the example shown, the side door protection system 200 is implemented within the vehicle 10. However, the side door protection system 200 can be implemented in any other propulsion system, such as, but not limited to, motorcycles, trucks, off-road vehicles, agricultural equipment, trains, airplanes, etc. The vehicle 10 includes data processing hardware 12 and memory hardware 14 storing instructions, which, when executed on the data processing hardware 12, cause the data processing hardware 12 to perform operations. The vehicle 10 also includes a user interface 30 with a display 32. Figure 5 and 6 The display 32 is configured to display a notification 232, warning the user of vehicle 10 that there is not enough space to open door 20. The user interface 30 may be implemented in the infotainment system of vehicle 10; however, it should be understood that the user interface 30 may be implemented in other computing devices (e.g., computing devices communicating with vehicle 10), such as, but not limited to, head-up displays, smartphones, tablets, smart displays, desktop / laptop computers, smartwatches, smart devices, or smart glasses / headsets.

[0039] Vehicle 10 may also include sensor systems 16, 16a-16f, configured to capture sensor data 202 from the environment of vehicle 10. Figure 2As shown in the figure, sensor system 16 includes a first front sensor 16a, a second front left sensor 16b, a third front right sensor 16c, a fourth rear left sensor 16d, a fifth rear right sensor 16e, and a sixth rear sensor 16f. However, it should be understood that sensor system 16 can include any number of sensors 16 at any location on vehicle 10. Note that sensor system 16 can include existing time-of-flight (TOF) sensors of vehicle 10, rather than additional sensors and / or devices that increase the weight, price, and computational complexity of vehicle 10. Here, sensor system 16 (i.e., TOF sensors) can be universally applied to models at all price points manufactured by the manufacturer of vehicle 10.

[0040] In some implementations, sensor system 16 is configured to capture sensor data 202 within an approach area 70 surrounding the exterior of vehicle 10. As used herein, approach area 70 may include the maximum range of objects (e.g., other vehicles, structures, people, etc.) that sensor 16 can accurately sense around vehicle 10. Although sensor system 16 includes discrete sensors around the exterior of vehicle 10, the sensor data 202 captured by sensor system 16 may collectively cover a 360-degree area surrounding vehicle 10. Vehicle 10 also includes four (4) doors 20, 20a-20d configured to allow users of vehicle 10 to enter and exit vehicle 10. Although vehicle 10 includes four (4) doors, side door protection system 200 may be implemented in vehicles with any number of doors, such as, but not limited to, two (2), three (3), four (4), etc.

[0041] The remote system 60 (e.g., a server, cloud computing environment) also includes data processing hardware 62 and memory hardware 64 storing instructions that, when executed on the data processing hardware 62, cause the data processing hardware 62 to perform operations. In some embodiments, the execution of the side door protection system 200 is shared by the vehicle 10 and / or the remote system 60. In some embodiments, the vehicle 10 and / or the remote system 60 execute the side door protection system 200 when the vehicle 10 is parked. Alternatively or concurrently, the vehicle 10 and / or the remote system 60 execute the side door protection system 200 when the vehicle 10 executes the parking procedure 400. As used herein, the parking procedure 400 can refer to any movement of the vehicle 10 to park, such as forward, reverse, parallel, etc. In some embodiments, the side door protection system 200 detects the parking procedure 400 based on the vehicle gear 22 of the vehicle 10 (e.g., park, drive, reverse).

[0042] refer to Figure 1 and Figure 2The side door protection system 200 executes a depth model 210, an area evaluator 220, and an alarm generator 230. In some embodiments, the side door protection system 200 may access a user data storage 240, which records / stores multiple regions of interest 18, 18a-18d of the vehicle 10, as well as any sensor data 202 collected / recorded during previous time steps. The user data storage 240 may be stored on any of the memory hardware 14, 64. In some embodiments, the side door protection system 200 is activated when it detects that a specific vehicle gear 22 of the vehicle 10 indicates that the vehicle 10 is performing a parking procedure 400. For example, the side door protection system 200 may be activated when the vehicle gear 22 of the vehicle 10 is reverse (i.e., the vehicle 10 is performing a parking procedure 400 of reversing into a parking space 402).

[0043] refer to Figure 2 and Figure 4 Depth model 210 is configured to receive sensor data 202 collected from a proximity area 70 surrounding vehicle 10 and generate a depth map 212 of the proximity area 70 as output. Specifically, depth model 210 is configured to estimate the location of object 404 within the proximity area 70 based on the sensor data 202. Here, depth model 210 may continuously or periodically receive sensor data 202 captured by sensor system 16 and measure the distance between vehicle 10 and any object 404 within proximity area 70. Depth model 210 may then compile the measurements derived from sensor data 202 and generate a depth map 212 estimating the position of object 404 relative to vehicle 10. In some embodiments, depth model 210 receives sensor data 202 and generates depth map 212 when vehicle 10 is parked (e.g., vehicle gear position 22 indicates vehicle 10 is parked). Optionally, when the vehicle 10 moves (e.g., when parking procedure 400 is executed), the depth model 210 can continuously receive sensor data 202 and use the continuously measured distance data to plot the distribution of objects 404 located near the door 20 to generate a depth map 212. In these cases, the depth model 210 can record historical sensor data 202 indicating the distance between the vehicle 10 and the objects 404 (e.g., in user data storage 240).

[0044] In some embodiments, depth model 210 additionally receives vehicle data 24 from vehicle 10. For example, vehicle data 24 may include one or more of the vehicle 10's position, vehicle 10's steering data, or vehicle 10's speed. Here, when depth model 210 generates depth map 212, depth model 210 can make vehicle data 24 consistent with sensor data 202 captured by sensor system 16 of vehicle 10. For example, the vehicle 10's position, vehicle 10's steering data, and vehicle 10's speed can be used to identify a baseline of the vehicle 10's position at any given time. The depth model can then correct the distance values ​​derived from sensor data 202 by sequentially reflecting changes in the vehicle 10's position (e.g., using vehicle data 24). In some embodiments, vehicle data 24 can be used to compensate for measured distances derived from sensor data 202.

[0045] Refer again Figure 2 The region evaluator 220 is configured to receive a depth map 212 including one or more objects 404 in the proximity region 70 of the vehicle 10 as input, and to generate an indication 222 as output whether the object 404 is located within the region of interest 18 corresponding to a door 20 of the vehicle 10. As used herein, the region of interest 18 of each door 20 of the vehicle 10 may include a three-dimensional space outside the door 20 in which surrounding objects 404 can be detected as interfering with the opening of the door 20.

[0046] Specific reference Figure 3A and 3B An example door 20 and its corresponding region of interest 18 are shown. Specifically, Figure 3A A top view of vehicle 10 is shown, wherein the region of interest 18 can be defined by the depth D through which the door 20 can swing. 18 And the length L, which is usually defined by the length of the door 20. 18 To limit. In addition, such as Figure 3B As shown, it displays a side perspective view of vehicle 10, with the area of ​​interest also including the height H, which is typically defined by the height of the door 20. 18 The length L of region of interest 18 18 Depth D of region of interest 18 18 And the height H of region of interest 18 18 It can be configured during the configuration / manufacturing process (e.g., initialization) of vehicle 10. For example, the depth D of region of interest 18. 18 It can be configured to include a default depth D that allows the door 20 to open to allow an adult to exit or enter the vehicle 10 with minimal space. 18 Here, the depth D of the region of interest is... 18 This can include the thickness of the door 20 plus 400 mm. In some embodiments, the depth D of the region of interest 18...18 It is adjustable. For example, the user of vehicle 10 can adjust the depth D of the region of interest 18. 18 To increase or decrease the depth D of region of interest 18 18 For example, a user can adjust the depth D of the region of interest 18 using the user interface 30 of vehicle 10. 18 .

[0047] Refer again Figure 2 The region estimator 220 receives the region of interest 18 corresponding to the door 20 of the vehicle 10 and determines whether there are any objects 404 within the region of interest 18 of the vehicle 10. In other words, the region estimator 220 determines, based on the depth map 212 and the three-dimensional region of interest 18, whether the distance between objects 404 in the depth map 212 is less than the relevant dimension (i.e., height H) in the region of interest 18. 18 Length L 18 Depth D 18 For each region of interest 18 of vehicle 10, region evaluator 220 can determine whether region of interest 18 is free of object 404. In some embodiments, indication 222 generated by region evaluator 220 specifies which doors 20 of vehicle 10 have object 404 in their corresponding regions of interest 18, and which doors 20 do not have object 404 in their corresponding regions of interest 18, and therefore can be opened.

[0048] Alarm generator 230 can receive an instruction 222 generated by area evaluator 220 as input and generate a notification 232 as output to the user of vehicle 10. Here, notification 232 can warn the user that a specific door 20 of vehicle 10 cannot be opened. Notification 232 can warn the user that more than one door 20 cannot be opened at the same time. In addition, notification 232 can include an instruction to the user indicating which door 20 is unobstructed (i.e., not blocked by anything) and can be opened at that time.

[0049] The alarm generator 230 can also receive seat data 26, indicating whether a seat associated with a door 20 is occupied, as input. Note that the side door protection system 200 can generate a notification 232 only for doors 20 adjacent to occupied seats. The seat data 26 may include data captured by a seat weight sensor or a seatbelt reminder system. Here, the alarm generator 230 can receive the seat data 26 and determine which doors 20 are associated with occupied seats. When the seat data 26 indicates that an occupied seat is adjacent to a door 20, the alarm generator 230 can include the occupied seat and the corresponding door 20 in the notification 232.

[0050] In some implementations, the notification 232 to the user is displayed on the display 32 of the user interface 30 of the vehicle 10. See details. Figure 5 The notification 232 may include images of vehicle 10 and objects 404a, 404b within the proximity area 70 surrounding vehicle 10. For example, the alarm generator 230 may generate a virtual image based on a depth map 212 generated by depth model 210, showing graphics of vehicle 10 and objects 404a, 404b within the proximity area 70. As shown, objects 404a, 404b (e.g., other vehicles) are displayed on either side of vehicle 10. Here, object 404a on the side of vehicle 10 may be parked such that it intrudes into the region of interest 18 of vehicle 10, while object 404b may be within the proximity area 70 but not within any region of interest 18 of vehicle 10. In this example, the side door protection system 200 may detect object 404a within the region of interest 18c corresponding to the rear passenger door 20c via sensor data 202. Furthermore, the side door protection system 200 can receive seat data 26 indicating that the rear passenger seats are occupied. Therefore, the side door protection system 200 can generate a notification 232, which includes an indication that the rear passenger door 20c corresponding to the region of interest 18c cannot be opened. The notification 232 may also include an indication "NOK," indicating that the rear passenger door 20c cannot be opened. Additionally, because the side door protection system 200 does not detect any objects in the regions of interest 18a, 18b corresponding to the front doors 20a, 20b of the vehicle 10 via sensor data 202, and seat data 26 indicates that the seats in front of the vehicle 10 are occupied, the notification 232 may include an indication that the front doors 20a, 20b can be opened and display "OK," indicating that the front doors can be opened. Conversely, although the region of interest 18d corresponding to the rear passenger door 20d may not include any objects, seat data 26 indicates that the seat is empty. Therefore, the notification 232 may exclude the rear passenger door 20d, or include visual exclusion (e.g., grayscale) on the rear passenger door 20d.

[0051] In some implementations, generating notification 232 to the user of vehicle 10 also includes additional safety measures to prevent the unlocked door 20 from being opened. For example, notification 232 may include an instruction to the door 20 to lock it and prevent it from opening. Here, locking the door 20 can attract the user's attention and give the user time to focus on object 404 in the area of ​​interest 18. In this example, the door 20 can be unlocked when the handle is pulled once. Optionally, notification 232 may include generating visual alerts for the user of the vehicle, such as turning lights on / off and / or flashing lights, to attract the user's attention. Alternatively or additionally, notification 232 may include an audio alert (such as a bell or synthesized voice warning) to convey that one or more doors 20 cannot be opened.

[0052] In some cases, the alarm generator 230 also detects the vehicle gear 22 of the vehicle 10 and generates the notification 232 only when the vehicle gear 22 indicates that the vehicle 10 is stopped. Conversely, refer to... Figure 4 and 6 The side door protection system 200 can be activated while the vehicle 10 is moving. For example, when the vehicle 10 is performing a parking procedure 400, the alarm generator 230 can display a notification 232 that highlights the area of ​​interest 18 of the vehicle 10 in real time to alert the user of the vehicle 10 to any object 404 that may prevent the door 0 from opening when the vehicle is parked.

[0053] Specific reference Figure 4 The figure illustrates a parking procedure 400 for vehicle 10. As shown, parking procedure 400 includes vehicle 10 reversing into parking space 402 positioned between adjacent objects 404a, 404b (i.e., other vehicles). However, it should be understood that parking procedure 400 may include forward parking and parallel parking. As vehicle 10 moves to park in parking space 402, sensor system 16 continuously scans the approach area 70 of vehicle 10 and captures sensor data 202 within the approach area 70. For example, as parking procedure 400 enters parking space 402, depth model 210 may iteratively update depth map 212 of parking space 402, starting with sensor data 202 collected from the rear corner of vehicle 10 and ending with sensor data 202 collected from the side door. Depth model 210 may identify objects 404a, 404b and include objects 404a, 404b in depth map 212. Subsequently, the region evaluator 220 can determine the trajectory of the vehicle 10 (e.g., based on vehicle data 24) to place the object 404a within the region of interest 18 of one of the doors 20 of the vehicle 10.

[0054] refer to Figure 6 When the vehicle performs parking procedure 400, alarm generator 230 can generate an indication 234 that highlights the region of interest 18 of the vehicle 10. For example, as shown, indication 234 can be displayed on display 32 of user interface 30. Specifically, indication 234 overlays the existing rearview camera image 602. Indication 234 can show the region of interest 18 proportional to the image 602 captured by the rearview camera to provide distance guidance of the vehicle 10 to the user of the vehicle 10 during parking procedure 400. Similar to... Figure 5Instruction 234 can exclude the region of interest 18d corresponding to the rear passenger door 20d with an empty seat, and highlight the regions of interest 18a, 18b corresponding to the front doors 20a, 20b as openable. Similarly, instruction 234 highlights the region of interest 18c corresponding to door 20c as containing object 404 (i.e., vehicle 404a). Advantageously, instruction 234 can provide the user with necessary advance notice to modify the parking procedure 400 by changing the trajectory of vehicle 10, restart the parking procedure 400, or find an alternative parking space that does not encroach on parking space 402 with object 404.

[0055] Figure 7 This includes a flowchart of an example operational setup for a method 700 for preventing side door accidents using existing TOF sensors. See also... Figure 1-6 To describe method 700. Data processing hardware (e.g. Figure 1 Data processing hardware 12, 62) can execute data stored in memory hardware (e.g., Figure 1 Instructions on the memory hardware (14, 64) are used to execute the example operation arrangement of method 700.

[0056] In operation 702, method 700 includes receiving sensor data 202 captured by sensor system 16 of vehicle 10. Method 700 also includes, in operation 704, mapping sensor data 202 to generate a depth map 212 of a proximity region 70 around vehicle 10. Here, depth map 212 includes object 404 in the proximity region 70 of vehicle 10. In operation 706, method 700 may further include determining whether object 404 in proximity region 70 is within a region of interest 18 corresponding to door 20 of vehicle 10. When object 404 is within region of interest 18 of vehicle 10, method 700 also includes, in operation 708, generating a notification 232 for a user of vehicle 10, the notification 232 warning the user that door 20 cannot be opened.

[0057] Figure 8 A flowchart of another example operational arrangement for method 800, which includes using existing TOF sensors to prevent side door accidents, is provided. Method 800 can be referenced. Figure 1-6 The description is executed when vehicle 10 performs parking procedure 400. Data processing hardware (e.g.) Figure 1 Data processing hardware 12, 62) can execute data stored in memory hardware (e.g., Figure 1 Instructions on the memory hardware (14, 64) are used to execute the example operation arrangement of method 800.

[0058] In operation 802, method 800 includes receiving sensor data 202 captured by sensor system 16 of vehicle 10. In operation 804, method 800 further includes mapping sensor data 202 to generate a depth map 212 of proximity region 70 around vehicle 10. Here, depth map 212 includes object 404 in proximity region 70 around vehicle 10. In operation 806, method 800 further includes determining whether object 404 in proximity region 70 is within region of interest 18 corresponding to door 20 of vehicle 10. When object 404 is within region of interest 18 of vehicle 10, method 800 further includes, in operation 808, displaying notification 232 highlighting region of interest 18 on display 32 communicating with data processing hardware 12 of vehicle 10.

[0059] Many embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other embodiments are also within the scope of the following claims.

[0060] The foregoing description has been provided for purposes of illustration and description. It is not intended to be exhaustive or limiting of this disclosure. Individual elements or features of a particular configuration are generally not limited to that particular configuration, but where applicable, they are interchangeable and can be used in selected configurations, even if not specifically shown or described. This can also be varied in many ways. Such variations should not be considered as departing from this disclosure, and all such modifications are intended to be included within the scope of this disclosure.

Claims

1. A computer-implemented method, when executed on data processing hardware, causes the data processing hardware to perform operations including: Receive sensor data captured by the vehicle's sensor system; Map sensor data to generate a depth map of the proximity area around the vehicle, the depth map including objects in the proximity area around the vehicle; Determine whether an object in the proximity area is within the region of interest corresponding to the car door; as well as When an object is within the area of ​​interest of the car door, a notification is generated for the vehicle's user, warning the user that the car door cannot be opened.

2. The method according to claim 1, wherein, The operation also includes receiving vehicle data, which includes one or more of the following: Vehicle location; Vehicle turning; or Vehicle speed.

3. The method according to claim 2, wherein, Mapping sensor data to generate a depth map of the proximity area around the vehicle involves aligning the vehicle's vehicle data with the sensor data captured by the vehicle's sensor system.

4. The method according to claim 1, wherein, The operation also includes receiving seat data indicating that a seat associated with a door is occupied.

5. The method according to claim 4, wherein, Generating a notification to warn users that the car door cannot be opened includes identifying the area of ​​interest corresponding to the occupied seat.

6. The method according to claim 1, wherein, Generating notifications for vehicle users includes displaying the notification on a display that communicates with the data processing hardware.

7. The method according to claim 6, wherein, Generating notifications to vehicle users also includes one or more of the following: Lock the car doors that cannot be opened; Generate visual alerts; or Generate an audio alarm.

8. The method according to claim 1, wherein, The region of interest is configured during the configuration process.

9. The method according to claim 1, wherein, The size of the region of interest is adjustable.

10. The method according to claim 1, wherein, The operation also includes detecting the vehicle's gear position and generating the notification for the vehicle's user when the gear position indicates that the vehicle should stop.

Images