Projected safety zone

Abstract

A system for providing a projected safety zone adjacent to a vehicle includes: an external projector in communication with a system controller and adapted to project a first graphic defining a predetermined safety zone onto a surface adjacent to the vehicle; and a plurality of in-vehicle sensors in communication with a system controller adapted to provide an alert to an individual and adapted to detect the presence of an individual located within the projected safety zone.

Description

Technical Field

[0001] The present disclosure relates to a system and method for providing a warning to an individual located within a safety zone projected onto the ground adjacent to a vehicle. Background Art

[0002] Vehicles are equipped with many sensors to monitor the environment around the vehicle and provide warnings when objects are approaching, etc. Current sensor systems for vehicles are not suitable for actively projecting a safety zone adjacent to the vehicle and for providing warnings to individuals who are within the safety zone and do not have credentials authorizing the individual to be within the safety zone.

[0003] Therefore, while current systems and methods achieve their intended purposes, a new and improved system and method is needed for projecting a safety zone adjacent to a vehicle and providing a warning to an individual when the individual is within the safety zone. Summary of the invention

[0004] According to several aspects of the present disclosure, a method for providing a projected safety zone adjacent to a vehicle includes: projecting a first graphic onto a surface adjacent to the vehicle using an external projector in communication with a system controller, the first graphic defining a predetermined safety zone; detecting the presence of an individual within the projected safety zone using a plurality of onboard sensors in communication with the system controller; and providing a warning to the individual using the system controller.

[0005] According to another aspect, the method further includes determining, using a system controller, whether the individual has a certificate authorizing the individual to be within the projected safety zone, and wherein providing a warning to the individual using the system controller further includes providing a warning to the individual using the system controller when the individual does not have a certificate authorizing the individual to be within the safety zone.

[0006] According to another aspect, determining, using a system controller, whether an individual has credentials authorizing the individual to be within a projected safety zone also includes accessing, using the system controller via a wireless communications module, a digital certificate of the individual, wherein the digital certificate is located in at least one of a cloud-based database and a personal device carried by the individual.

[0007] According to another aspect, determining, using a system controller, whether an individual has a credential authorizing the individual to be within a projected safety zone further comprises capturing an image of the individual using a camera in communication with the system controller, and identifying, using a computer vision algorithm within the system controller, a physical credential, wherein the physical credential comprises at least one of a facial identification, specific clothing, and a badge.

[0008] According to another aspect, the method further includes identifying, using a system controller, threatening behavior exhibited by an individual, and wherein, providing a warning to the individual using the system controller further includes providing a warning to the individual using the system controller when the individual has a certificate authorizing the individual to be within a safe zone, and the system controller identifies the threatening behavior exhibited by the individual.

[0009] According to another aspect, identifying threatening behavior exhibited by an individual using a system controller also includes receiving verbal communications from the individual using the system controller via an external microphone and identifying threatening language using the system controller and capturing an image of the individual using a camera in communication with the system controller and identifying at least one of threatening gestures performed by the individual using a computer vision algorithm within the system controller.

[0010] According to another aspect, providing a warning to the individual using the system controller further includes projecting a second graphic onto a surface adjacent to the vehicle using an external projector in communication with the system controller, the second graphic being color coded and highlighting a location of the individual.

[0011] According to another aspect, providing a warning to the individual using the system controller further includes determining, using the system controller, that the individual is still within the safety zone after projecting the second graphic, and providing an audible warning to the individual via an external speaker.

[0012] According to another aspect, the method further includes providing, with the system controller, via the wireless communication module, a warning signal to a personal device carried by the individual.

[0013] According to another aspect, providing a warning to an individual using the system controller also includes, after providing an audible warning to the individual via an external speaker, determining using the system controller that the individual is still within a safe zone, and providing a tactile warning signal to a personal device carried by the individual via a wireless communication module using the system controller.

[0014] According to another aspect, the method further includes detecting, using a plurality of onboard sensors, when an occupant of the vehicle exits the vehicle, and wherein projecting the first graphic onto a surface adjacent to the vehicle using an external projector in communication with the system controller further includes projecting the first graphic onto a surface adjacent to the vehicle and centered on an occupant of the vehicle using the external projector.

[0015] According to another aspect, providing a warning to a personal device carried by an individual using a system controller via a wireless communication module also includes providing a warning message to a personal device carried by an occupant using the system controller via a wireless communication module, and providing a tactile warning signal to a personal device carried by an individual using the system controller via a wireless communication module also includes providing a tactile warning signal to the personal device carried by the occupant using the system controller via a wireless communication module.

[0016] According to another aspect, the method further includes receiving, using a system controller, instructions from an occupant via a wireless communication module to provide a warning to an individual within the projected safety zone, and wherein providing a warning to an individual using the system controller further includes providing a warning to an individual using the system controller when the instructions are received from the occupant.

[0017] According to several aspects of the present disclosure, a system for providing a projected safety zone adjacent to a vehicle includes an external projector that communicates with a system controller and is adapted to project a first graphic defining a predetermined safety zone onto a surface adjacent to the vehicle, and a plurality of onboard sensors that communicate with the system controller and are adapted to detect the presence of an individual within the projected safety zone, the system controller being adapted to provide a warning to the individual.

[0018] According to another aspect, the system controller is further adapted to determine whether the individual has a credential authorizing the individual to be within the projected safety zone by at least one of: utilizing the system controller to access a digital credential of the individual via a wireless communication module, wherein the digital credential is located within at least one of a cloud-based database and a personal device carried by the individual; and utilizing a camera in communication with the system controller to capture an image of the individual and utilizing a computer vision algorithm within the system controller to identify a physical credential, wherein the physical credential includes at least one of a facial identification, specific clothing, and a badge, wherein the system controller is adapted to provide a warning to the individual when the individual does not have a credential authorizing the individual to be within the safety zone.

[0019] According to another aspect, the system controller is further adapted to identify threatening behavior exhibited by the individual by at least one of: receiving verbal communications from the individual via an external microphone using the system controller, identifying threatening language using the system controller, and capturing an image of the individual using a camera in communication with the system controller, and identifying threatening gestures made by the individual using a computer vision algorithm within the system controller, and wherein the system controller is further adapted to provide a warning to the individual when the individual has a certificate authorizing the individual to be within a safe zone, and the system controller identifies threatening behavior exhibited by the individual.

[0020] According to another aspect, when providing a warning to an individual, the system controller is further adapted to project a second graphic onto a surface adjacent to the vehicle using an external projector in communication with the system controller, the second graphic being color-coded and highlighting the location of the individual, determining that the individual is still within a safe zone after projecting the second graphic, providing an audible warning to the individual via an external speaker and providing a warning signal to a personal device carried by the individual via a wireless communication module, and determining that the individual is still within a safe zone after providing the audible warning and providing a tactile warning signal to a personal device carried by the individual via the wireless communication module.

[0021] According to another aspect, the system controller is further adapted to detect when an occupant within the vehicle leaves the vehicle using a plurality of onboard sensors and project a first graphic centered on the occupant of the vehicle, receive instructions from the occupant via the wireless communication module to provide warnings to individuals within the projected safety zone, and provide warnings to individuals when instructions are received from the occupant.

[0022] According to another aspect, when providing a warning signal to a personal device carried by an individual, the system controller is also suitable for providing a warning signal to a personal device carried by an occupant via a wireless communication module, and when providing a tactile warning signal to a personal device carried by an individual, the system controller is also suitable for providing a tactile warning signal to the personal device carried by the occupant via the wireless communication module.

[0023] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter.It should be understood that the detailed description and specific examples are intended for illustration purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0025] Figure 1 is a schematic diagram of a vehicle having a system for projecting a safety zone and providing a warning to an individual located within the safety zone according to an exemplary embodiment;

[0026] Figure 2 is a schematic diagram of a system according to an exemplary embodiment;

[0027] Figure 3 is a schematic perspective view of an occupant standing near a vehicle within a projected safety zone;

[0028] Figure 4 is a schematic perspective view of a wheelchair-bound occupant about to enter a vehicle that is projecting a safety zone;

[0029] Figure 5 is a schematic side view of a vehicle projecting a safety zone adjacent to an open door of the vehicle;

[0030] Figure 6 is a schematic perspective view of a vehicle in a construction zone adjacent to a safety zone projected by the vehicle;

[0031] Figure 7 is a schematic perspective view of a vehicle, the vehicle being a heavy equipment vehicle adjacent to a safety zone projected by the vehicle, wherein an individual with a certificate is standing adjacent to the vehicle within the safety zone;

[0032] Figure 8is a schematic perspective view of two bystanders approaching a projected safety zone behind a vehicle about to exit a parking space;

[0033] Fig. 9 is a graphic representation of the hands of an individual being analyzed for threatening behavior by a system controller;

[0034] Fig.10 is a flow chart for analyzing the spatiotemporal characteristics of a person's physical attributes using a system controller; and

[0035] Fig.11 is a flow chart of a method according to an exemplary embodiment of the present disclosure.

[0036] These figures are not necessarily to scale, and some features may be enlarged or reduced, such as to illustrate details of a particular component. In some cases, well-known components, systems, materials, or methods are not described in detail to avoid obscuring the present disclosure. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art to use the present disclosure in various ways. DETAILED DESCRIPTION

[0037] The following description is essentially only exemplary and is not intended to limit the present disclosure, application or use. In addition, it is not desired to be bound by any express or implied theory presented in the aforementioned technical field, background technology, summary of the invention or the following specific embodiments. It should be understood that in all drawings, the corresponding reference numerals represent the same or corresponding parts and features. As used herein, the term module represents any hardware, software, firmware, electronic control component, processing logic and / or processor device individually or in any combination, including but not limited to: specific integrated circuit (ASIC), electronic circuit, processor (shared, specific or group) and memory, combinational logic circuit and / or other suitable components providing the function that execute one or more software or firmware programs. Although the drawings shown in this article describe examples with certain element arrangements, additional intermediate elements, devices, features or components may be present in actual embodiments. It should also be understood that the drawings are merely illustrative and may not be drawn to scale.

[0038] As used herein, the term "vehicle" is not limited to automobiles. Although the present technology is described primarily in conjunction with automobiles herein, the present technology is not limited to automobiles. These concepts can be used in a variety of applications, such as in connection with aircraft, ships, other vehicles, and consumer electronic components.

[0039] According to an exemplary embodiment, Figure 1A vehicle 10 is shown with an associated system 11 for providing a warning to an occupant 56 within the vehicle 10 to notify the occupant 56 of a threatening vehicle when the vehicle 10 is at an intersection, according to various embodiments. Typically, the system 11 works with other systems within the vehicle 10 to display various information and infotainment content to the occupant. The vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially surrounds the components of the vehicle 10. The body 14 and the chassis 12 may together form a vehicle frame. The front wheels 16 and the rear wheels 18 are each rotatably coupled to the chassis 12 near respective corners of the body 14.

[0040] In various embodiments, the vehicle 10 is an autonomous vehicle, and the system 11 is incorporated into the autonomous vehicle 10. The autonomous vehicle 10 is, for example, a vehicle 10 that is automatically controlled to transport passengers from one location to another. The vehicle 10 is depicted as a passenger car in the illustrated embodiment, but it should be understood that any other vehicle including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), etc. may also be used. In an exemplary embodiment, the vehicle 10 is equipped with a so-called level four or level five automation system. The level four system represents "high automation" and refers to the driving mode-specific performance of the autonomous driving system in all aspects of the dynamic driving task even if the human driver does not respond appropriately to the intervention request. The level five system represents "full automation" and refers to the full-time performance of the autonomous driving system in all aspects of the dynamic driving task under all road and environmental conditions that can be managed by the human driver. The novel aspects of the present disclosure are also applicable to non-autonomous vehicles.

[0041] As shown, the vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, a vehicle controller 34, and a wireless communication system 36. In embodiments where the vehicle 10 is an electric vehicle, there may be no transmission system 22. In various embodiments, the propulsion system 20 may include an internal combustion engine, an electric machine such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transfer power from the propulsion system 20 to the front wheels 16 and rear wheels 18 of the vehicle according to a selectable speed ratio. According to different embodiments, the transmission system 22 may include a stepped automatic transmission, a continuously variable transmission, or an appropriate transmission. The braking system 26 is configured to provide braking torque to the front wheels 16 and rear wheels 18 of the vehicle. In various embodiments, the braking system 26 may include friction brakes, line brakes, regenerative braking systems such as electric machines, and / or other appropriate braking systems. The steering system 24 affects the position of the front wheels 16 and rear wheels 18. Although depicted as including a steering wheel for purposes of illustration, in some embodiments contemplated within the scope of the present disclosure, the steering system 24 may not include a steering wheel.

[0042] The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external environment and / or the internal environment of the autonomous vehicle 10. The sensing devices 40a-40n may include, but are not limited to, radars, lidars, global positioning systems, optical cameras, thermal imagers, ultrasonic sensors, and / or other sensors. The camera may include two or more digital cameras spaced apart from each other by a selected distance, wherein the two or more digital cameras are used to obtain stereo images of the surrounding environment in order to obtain a three-dimensional image or map. The plurality of sensing devices 40a-40n are used to determine information about the surrounding environment of the vehicle 10. In an exemplary embodiment, the plurality of sensing devices 40a-40n include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor. In another exemplary embodiment, the plurality of sensing devices 40a-40n also include sensors for determining information about the environment surrounding the vehicle 10, for example, an ambient air temperature sensor, an atmospheric pressure sensor, and / or a photo and / or video camera positioned to view the environment in front of the vehicle 10. In another exemplary embodiment, at least one of the plurality of sensing devices 40 a - 40 n is capable of measuring distances in the environment surrounding the vehicle 10 .

[0043] In a non-limiting example where the plurality of sensing devices 40a-40n include a camera 38, the plurality of sensing devices 40a-40n measure distance using an image processing algorithm configured to process images from the camera and determine distances between objects. In another non-limiting example, the plurality of vehicle sensors 40a-40n include a stereo camera with distance measurement capabilities. In one example, at least one of the plurality of sensing devices 40a-40n is fixed to the interior of the vehicle 10, such as in the roof of the vehicle 10, with a view through the windshield of the vehicle 10. In another example, at least one of the plurality of sensing devices 40a-40n is a camera fixed to the exterior of the vehicle 10 (e.g., on the roof of the vehicle 10), having a view of the environment surrounding the vehicle 10 and adapted to collect information (images) related to the environment outside the vehicle 10. It should be understood that various additional types of sensing devices, such as LiDAR sensors, ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors are within the scope of the present disclosure. The actuator system 30 includes one or more actuator devices 42 a - 42 n that control one or more features of the vehicle 10 , such as, but not limited to, the propulsion system 20 , the transmission system 22 , the steering system 24 , and the braking system 26 .

[0044] The vehicle controller 34 includes at least one processor 44 and a computer readable storage device or medium 46. The at least one data processor 44 can be any custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor of several processors associated with the vehicle controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, any combination thereof, or any device generally used to execute instructions. The computer readable storage device or medium 46 can include volatile and non-volatile storage in, for example, read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the at least one data processor 44 is powered off. The computer readable storage device or medium 46 can be implemented using any of a variety of known memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combination memory device capable of storing data, some of which represents executable instructions used by the controller 34 in controlling the vehicle 10.

[0045] The instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. When executed by at least one processor 44, the instructions receive and process signals from the sensor system 28, execute logic, calculations, methods and / or algorithms for automatically controlling components of the vehicle 10, and generate control signals to the actuator system 30 based on the logic, calculations, methods and / or algorithms to automatically control components of the vehicle 10. Although in Figure 1 Only one controller 34 is shown, but embodiments of vehicle 10 may include any number of controllers 34 that communicate via any suitable communication medium or combination of communication mediums and cooperate to process sensor signals, execute logic, calculations, methods and / or algorithms, and generate control signals to automatically control features of autonomous vehicle 10.

[0046] In various embodiments, one or more instructions of the vehicle controller 34 are included in the trajectory planning system and, when executed by at least one data processor 44, generate trajectory outputs that solve the kinematic and dynamic constraints of the environment. For example, the instructions receive process sensor and map data as input. The instructions execute a graph-based approach with a customized cost function to handle different road scenarios in urban and highway roads.

[0047] The wireless communication system 36 is configured to wirelessly communicate with other remote entities 48, such as, but not limited to, other vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems, remote servers, cloud computers, and / or personal devices. In an exemplary embodiment, the communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communications. However, additional or alternative communication methods such as a specific short-range communication (DSRC) channel are also considered to be within the scope of the present disclosure. A DSRC channel refers to a one-way or two-way short-range to medium-range wireless communication channel specific to automotive use and a corresponding set of protocols and standards.

[0048] The vehicle controller 34 is a non-general purpose electronic control device having a pre-programmed digital computer or processor, a memory or non-transitory computer readable medium for storing data such as control logic, software applications, instructions, computer code, data, look-up tables, etc., and a transceiver [or input / output port]. Computer readable media include any type of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard disk drive, compact disk (CD), digital video disk (DVD), or any other type of memory. "Non-transitory" computer readable media do not include wired, wireless, optical or other communication links that transmit temporary electrical or other signals. Non-transitory computer readable media include media in which data can be permanently stored and media in which data can be stored and subsequently rewritten, such as rewritable optical disks or erasable memory devices. Computer code includes any type of program code, including source code, object code, and executable code.

[0049] refer to Figure 2 , a schematic diagram of the system 11 is shown. The system 11 includes a system controller 34A in communication with a plurality of sensing devices (on-board sensors) 40a-40n. The system controller 34A may be the vehicle controller 34, or the system controller 34A may be a separate controller in communication with the vehicle controller 34. As described above, the system controller 34A communicates with the plurality of on-board sensors 40a-40n, including at least one camera 38 adapted to capture images of the vehicle's surroundings and objects therein. In addition to the plurality of on-board sensors 40a-40n, the system controller 34A also communicates with an external microphone 50, an external speaker 52, and an external projector 54.

[0050] refer to Figure 3 In the exemplary embodiment, the system controller 34A is adapted to project a first graphic 56 onto a surface adjacent to the vehicle 10 using an external projector 54. The surface adjacent to the vehicle 10 may be a ground surface, a parking lot, a road, or any other surface on which the vehicle 10 is located. The external projector 54 is of any known type suitable for projecting an image onto a surface adjacent to the vehicle 10. The first graphic 56 defines a predetermined safety zone 58.

[0051] Safety zones 58 projected onto surfaces adjacent to vehicle 10 provide visible areas where people should stay outside. Figure 3As shown, the occupant 60 of the vehicle 10 may be a celebrity or politician, wherein when the occupant 60 leaves the vehicle 10, the safety zone 58 is intended to provide a visual barrier that bystanders 62 (fans) should remain outside of. In an exemplary embodiment, the system controller 34A is adapted to utilize a plurality of onboard sensors 40a-40n to detect when the occupant 60 in the vehicle 10 leaves the vehicle 10 and project the first graphic 56 centered on the occupant 60 of the vehicle 10. In another example, referring to Figure 4 , the safety zone 58 provides a visual barrier to keep bystanders 62 at a distance when the wheelchair occupant 60 enters or exits the vehicle 10. When the system controller 34A detects that the wheelchair occupant is about to enter or exit the vehicle 10, such as when the loading / unloading ramp / platform is extended, the system controller 34A projects the first graphic.

[0052] refer to Figure 5 In another example, the safety zone 58 provides a visual barrier to keep bystanders 62 (pedestrians, cyclists) away from the area immediately adjacent to the door 64 of the vehicle 10, which, if opened, could pose a risk as the bystanders 62 pass by when the door 64 of the vehicle 10 is open. In an exemplary embodiment, the system controller 34A projects the safety zone 58 when it detects that an occupant 60 in the vehicle 10 may open the door 64 and exit the vehicle 10. Figure 5 As shown, the first graphic 56 may also include text language 70 to also convey a message to the bystander 62, such as shown where the first graphic 56 includes both the safety zone 58 and the text language 70 stating "Open Door."

[0053] refer to Figure 6 and Figure 7 In another example, the safety zone 58 provides a visual barrier to keep people and vehicles away from areas within or near the construction zone. Figure 8 , the safety zone 58 provides a visual barrier to keep people away from the area behind the vehicle 10 when the vehicle 10 is about to move. As shown, the safety zone is projected behind the vehicle 10 to warn approaching bystanders 62 that the vehicle may be about to back out of a parking space, thereby posing a hazard to the approaching bystanders 62. In the exemplary embodiment, the safety zone is activated when the system controller 34A detects that the vehicle 10 has reversed and detects the presence of the approaching bystander 62.

[0054] In another exemplary embodiment, the system controller 34A will modify the safety zone 58 as the environment changes. For example, when the vehicle 10 is placed in reverse gear, the safety zone 58 is activated. Once the vehicle 10 begins to move, the system controller 34A can expand the safety zone 58 or change the color of the safety zone 58 to visually communicate the increased risk. For example, when the vehicle 10 is in reverse but not moving, the safety zone 58 can be projected as a yellow area, and when the vehicle 10 begins to move backward, the system controller 34A can expand the safety zone 58 and change the color to red. In addition, if the vehicle 10 is already moving and accelerating, the system controller 34A can expand or change the color of the safety zone 58.

[0055] In an exemplary embodiment, a plurality of onboard sensors 40a - 40n in communication with the system controller 34A are adapted to detect the presence of an individual 66 located within the projected safety zone 58 , and the system controller 34A is adapted to provide a warning to the individual.

[0056] In the exemplary embodiment, when providing a warning to the individual 66, the system controller 34A is further adapted to project a second graphic 68 onto a surface adjacent to the vehicle 10 using the external projector 54 in communication with the system controller 34A, the second graphic 68 being color-coded and highlighting the location of the individual 66. Referring again to FIG. Figure 3 , the system controller 34A detects the presence of the individual 66 (the person taking the photo of the occupant 60) within the projected safety zone 58. In response, the system controller 34A projects a second graphic 68 (a circular highlighted area) centered on the location of the individual 66 via the projector 54. The second graphic 68 is used to provide a visual indication to the individual 66 that the individual 66 has entered the safety zone 58. The second graphic 68 can be color coded, where, for example, when the individual 66 first enters the safety zone 58, the second graphic is yellow, but if the individual is still within the safety zone 58 or moves closer to the occupant 60, the second graphic can change from yellow to red. In addition, if the individual stays within the safety zone for more than a predetermined amount of time, the second graphic 68 can change from yellow to red. The system controller 34A can also cause the second graphic to change color of the pattern, flash and blink, or alternately become larger / larger to more effectively attract the attention of the individual 66.

[0057] In the exemplary embodiment, the system 11 includes a graphics engine 72 in communication with the system controller 34A. The graphics engine 72 uses data captured by the plurality of onboard sensors 40a-40n to determine the appropriate first graphics 56 and the appropriate second graphics 68 depending on the situation, how many people 66 are present within the safety zone, etc.

[0058] In an exemplary embodiment, the system controller 34A is further adapted to determine that the individual is still within the safety zone 58 after projecting the second graphic 68 and to provide an audible warning to the individual 66 via the external speaker 52. Figure 3 , if after projecting the second graphic 68, the system controller 34A determines using the plurality of onboard sensors 40a-40n that the individual has not moved out of the safety zone 58, the system controller 34A will provide an audible warning via the speaker 52. The audible warning may include a subtle click or beep broadcast by the speaker 52 and is used to draw attention and notify the individual 66 that the individual 66 has entered the safety zone 58. If the behavior of the individual 66 persists, the system controller 34A is adapted to increase the severity of the audible warning by, for example, increasing the volume of the audible warning, increasing the pitch of the audible warning, etc. Finally, if the behavior of the individual 66 escalates or persists for more than a predetermined period of time, the system controller 34A may use the speaker 52 to provide an audible warning that provides verbal instructions to the individual 66. For example, if Figure 3 As shown, in the case where fans have approached the celebrity, the speaker 53 may broadcast a voice warning of "please step back", or Figure 8 As shown, speaker 53 may broadcast an audible warning of "Stop!" to a couple approaching a vehicle exiting a parking space in front of them.

[0059] In another exemplary embodiment, if the system controller 34A determines that the individual 66 is still within the safety zone 58 after projecting the second graphic 68 and providing the audible warning to the individual 66, the system controller 34A is further adapted to provide a warning signal to the personal device 74 carried by the individual 66 via the wireless communication module 36. Thus, when the individual enters the safety zone 58, the system controller 34A, after recognizing the individual via the face, sends a signal to the mobile phone or other personal device carried by the individual 66 to notify the individual that they have entered the safety zone.

[0060] In another exemplary embodiment, the system controller 34A is further adapted to determine that the individual 66 is still within the safety zone 58 after providing the audible warning, and to provide a tactile warning signal to the personal device 74 carried by the individual 66 via the wireless communication module 36. Thus, if the individual 66 does not respond to the projection of the second graphic 68, does not respond to the broadcast of the audible warning, and does not respond to the warning signal sent to the personal device 74, the system controller 34A will send a signal to the personal device 74 causing the personal device to provide a tactile warning, such as a vibration, a beep, provide a verbal warning, etc.

[0061] In an exemplary embodiment, when providing a warning signal to the personal device 74 carried by the individual 66, the system controller 34A is also suitable for providing a warning signal to the personal device 76 carried by the occupant 60 via the wireless communication module 36, and when providing a tactile warning signal to the personal device 74 carried by the individual 66, the system controller 34A is also suitable for providing a tactile warning signal to the personal device 76 carried by the occupant 60 via the wireless communication module 36.

[0062] Thus, when a signal is sent to the personal device 74 of the individual 66, a signal is sent to the personal device 76 of the occupant 60. This informs the occupant 60 of the fact that such a signal is sent, so that the occupant knows when and how the individual 66 within the safety zone 58 is alerted, and draws the occupant's attention to the individual 66. Furthermore, in another exemplary embodiment, when a signal is sent to the personal device 74 of the individual, the system controller 34A may send such a signal to a pre-specified individual, such as a security guard or bodyguard, to alert them to the presence of the individual 66.

[0063] In an exemplary embodiment, system controller 34A is further adapted to determine whether individual 66 has credentials authorizing individual 66 to be within projected safety zone 58. System controller 34A is adapted to provide a warning to individual 66 when individual 66 does not have credentials authorizing individual 66 to be within safety zone 58.

[0064] The system controller 34A may determine whether the individual 66 is credentialed by accessing a digital credential of the individual 66 via the wireless communication module 36, wherein the digital credential is located in at least one of a cloud-based database 78 and a personal device 74 carried by the individual 66. The system controller 34A may also determine whether the individual 66 is credentialed by capturing an image of the individual 66 using a camera 38 in communication with the system controller 34A and identifying a physical credential using a computer vision algorithm 80 within the system controller 34A, wherein the physical credential includes at least one of a facial identification, a specific garment, and a badge.

[0065] Reference again Figure 3 As shown, the first graphic 56 projects a safety zone 58 that includes multiple layers, wherein the presence of an individual within each particular layer of the safety zone 58 is determined by the credentials they possess. Figure 3 As shown, the individual 66 (the photographer) may have appropriate credentials that allow the individual 66 to be within the first level 82 of the safety zone 58, but these credentials do not authorize the individual 66 to get closer to the occupant 60 by entering the second level 84 of the safety zone 58. Therefore, when the individual 66 moves from the first level 82 of the safety zone 58 to the second level 84 of the safety zone, the system controller 34A provides a warning to inform the individual 66 that they are too close to the occupant 60.

[0066] refer to Figure 7 , first graphic 56 defines a safety zone 58 adjacent to a piece of heavy equipment 86 to keep bystanders away. System controller 34A detects the presence of individual 66 within safety zone 58, however, individual 66 is wearing safety gear and color-coded clothing indicating that the individual is a construction worker. System controller 34A uses camera 38 and computer vision algorithm 80 to determine that individual 66 has credentials authorizing individual 66 to be within safety zone 58, and therefore does not provide a warning to individual 66.

[0067] In another exemplary embodiment, the system controller 34A is further adapted to identify threatening behavior exhibited by the individual 66 by at least one of: 1) capturing an image of the individual 66 using a camera 38 in communication with the system controller 34A and identifying threatening gestures made by the individual 66 using a computer vision algorithm 80 within the system controller 34A, and 2) receiving verbal communications from the individual 66 via the external microphone 50 using the system controller 34A and identifying threatening language using the system controller 34A. The system controller 34A is further adapted to provide a warning to the individual 66 when the system controller 34A identifies threatening behavior exhibited by the individual 66, even when the individual 66 has credentials authorizing the individual 66 to be present within the safety zone 58.

[0068] In an exemplary embodiment, when capturing images of individual 66 using a camera 38 in communication with system controller 34A, and identifying threatening gestures made by individual 66 using a computer vision algorithm 80 within system controller 34A, system controller 34A is adapted to identify physical characteristics of individual 66. Physical attributes include the position of individual 66, body movements, such as whether individual 66 is walking, running, crawling, lying, laughing, crying, reading, pushing, pulling, etc. System controller 34A also identifies physical attributes such as the position and dynamics of individual 66's legs, arms, and head. The two-dimensional and three-dimensional skeletal position and dynamics of individual 66 are located and tracked as vector representations. System controller 34A also tracks the motion dynamics of individual 66, such as direction, velocity, and acceleration, as well as the duration of any action (how long did individual 66 do this action?).

[0069] The system controller 34A is adapted to analyze the spatiotemporal characteristics of the physical characteristics of the individual 66. The system controller 34A is adapted to determine whether any physical characteristics of the individual 66 indicate threatening behavior of the individual 66. The system controller 34A is adapted to compare the physical characteristics of the individual 66 with a "normal" pattern, which is obtained from a remote database 88 of known patterns indicating threatening behavior, or from a neural network stored in the system controller 34A, which uses machine learning to compile a set of known patterns indicating threatening behavior. In addition to comparing the current pattern with the known pattern, other characteristics are used to determine whether the movement and action of the individual 66 should be interpreted as threatening behavior. For example, in a crowd, if the individual 66 makes a gesture or action, it can be interpreted as threatening behavior by the system controller 34A. However, if multiple individuals 66 in the group are making the same gesture or action, there is a higher possibility of threatening behavior. Time is always an important factor. The longer the action or activity lasts, the higher the possibility that such action or activity constitutes threatening behavior. Similarly, the distance from the vehicle 10 or the occupant 60 is an important consideration. The further away the individual 66 is from the vehicle 10 and / or occupants 60, the less likely it is that any behavior on the part of the individual 66 is dangerous. If the system controller 34A determines that the physical characteristics of the individual 66 do not indicate threatening behavior on the part of the individual 66, the system controller 34A is adapted to continue analyzing the continuously collected spatiotemporal characteristics of the physical characteristics of the individual 66.

[0070] refer to Fig. 9 and Fig.10 In another exemplary embodiment of the present disclosure, when analyzing the spatiotemporal characteristics of the physical characteristics of the individual 66 using the system controller 34A at 90, the system controller 34A is also adapted to identify key points 94 of the hand 96 of the individual 66 at 92. Fig. 9 , the key points 94 (joint locations within the hand and fingers, as shown) of the hand 96 of the individual 66 are highlighted and linked to allow the system controller 34A to determine the shape or gesture being displayed by the hand 96. As non-limiting examples, the system 11 may use mesh model matching, depth-based methods, deep learning-based post-estimation methods (such as open pose or deep pose), or multi-view guidance when creating the key point model of the hand 96.

[0071] At 98, the system controller 34A is further adapted to compare the identified key points 94 of the hand 96 of the individual 66 to known threatening gestures stored in the remote database 88. Alternatively, the system controller 34A may compare the identified key points 94 of the hand 96 of the individual 66 to known threatening gestures stored in a neural network stored in the system controller 34A that compiles a set of known threatening gestures using machine learning. At 100, if the system controller 34A determines that the identified key points 94 of the hand 96 of the individual 66 matches at least one of the known threatening gestures stored in the database 88, the system controller 34A is adapted to determine that the individual 66 is exhibiting threatening behavior, as indicated by line 102. If the system controller 34A determines that the identified key points 94 of the hand 96 of the individual 66 does not match at least one of the known threatening gestures stored in the database 88, the system controller 34A is adapted to continue comparing the identified key points 94 of the hand 96 of the individual 66 to the remote database 88 of known threatening gestures, as indicated by line 104.

[0072] In another exemplary embodiment, when the system controller 34A receives verbal communications from the individual 66 via the external microphone 50 and identifies threatening language using the system controller 34A, the system controller 34A is further adapted to extract features of the acoustic signal captured by the microphone 50 and compare the features of the acoustic signal to an optimized model of known sounds indicative of threatening behavior stored in the remote database 88. The system controller 34A is further adapted to determine that the individual 66 is exhibiting threatening behavior when the captured acoustic signal matches the known sounds from the optimized model. Speech and non-speech sounds indicative of threatening behavior include, but are not limited to, shouting, hits / hitting with drumsticks, banging or crashing sounds, swearing, insults, danger words, and bullying and taunting sounds such as a deep male voice with laughter / sadness.

[0073] The optimization model is created and updated in the remote database 88. Starting from a non-speech audio event dataset, the dataset includes data collected, augmented and grouped to take into account different noise factors such as season, weather conditions, location, background noise and reverberation, vehicle speed, etc. The data of the non-speech audio event dataset is processed to create a pre-processed non-speech audio event dataset. Distinguishing features of the pre-processed non-speech audio event dataset are identified and fed to training and testing datasets where model training / testing is performed, and a machine learning algorithm is used to optimize candidate models to create an optimized model of the identified dangerous sounds and language.

[0074] In an exemplary embodiment, an acoustic signal is captured by microphone 50, and system controller 34A extracts features of the acoustic signal, such as, but not limited to, Mel-Frequency Cepstral Coefficients (MFCC), Filterbank Energy, Log Filterbank Energy, and Spectral Subband Centroids. When these features match features in an optimized model of known dangerous sounds, system controller 34A determines that individual 66 is engaging in threatening behavior.

[0075] In yet another exemplary embodiment, the system controller 34A is further adapted to receive instructions from the occupant 60 via the wireless communication module 36 to provide a warning to the individual 66 within the projected safety zone 58, and to provide the warning to the individual 66 when the instruction is received from the occupant 60. Thus, the occupant may send instructions to the system controller 34A to project a warning to the individual 66 by using an application on the personal device 76 or by using a simple buckle actuator. This would override any determination by the system controller 34A that the individual 66 has the proper credentials or is otherwise permitted to be within the safety zone 58. The occupant 60 may selectively initiate the warning at any time that the occupant 60 feels a warning to the individual is needed.

[0076] refer to Fig.11 , a method 200 for providing a projected safety zone 58 adjacent to a vehicle 10 includes (starting at box 202) projecting a first graphic 56 onto a surface adjacent to the vehicle 10 using an external projector 54 in communication with a system controller 34A, the first graphic 56 defining a predetermined safety zone 58 (moving to box 204), detecting the presence of an individual 66 located within the projected safety zone 58 using a plurality of onboard sensors 40a-40n in communication with the system controller 34A, and moving to box 206, providing a warning to the individual 66 using the system controller 34A.

[0077] In an exemplary embodiment, method 200 also includes (moving to box 208) using the system controller 34A to determine whether individual 66 has a certificate authorizing individual 66 to be within the projected safety zone 58, and using the system controller 34A at box 206 to provide a warning to individual 66 also includes using the system controller 34A to provide a warning to individual 66 when individual 66 does not have a certificate authorizing individual 66 to be within the safety zone 58, as shown by line 210.

[0078] In an exemplary embodiment, determining at box 208 whether the individual 66 has a certificate authorizing the individual 66 to be within the projected safety zone 58 using the system controller 34A also includes using the system controller 34A to access a digital certificate of the individual 66 via the wireless communication module 36, wherein the digital certificate is located in at least one of a cloud-based database 78 and a personal device 74 carried by the individual 66.

[0079] In another exemplary embodiment, determining at box 208 whether the individual 66 has a credential authorizing the individual 66 to be within the projected safety zone 58 using the system controller 34A also includes capturing an image of the individual 66 using a camera 38 in communication with the system controller 34A, and identifying a physical credential using a computer vision algorithm 80 within the system controller 34A, wherein the physical credential includes at least one of a facial identification, specific clothing, and a badge.

[0080] In an exemplary embodiment, method 200 also includes (moving from box 208 to box 212) using the system controller 34A to identify threatening behavior performed by individual 66, wherein using the system controller 34A at box 206 to provide a warning to individual 66 also includes using the system controller 34A to provide a warning to individual 66 when individual 66 has a certificate authorizing individual 66 to be within the safe zone 58, as shown by line 214, and the system controller 34A identifies the threatening behavior exhibited by individual 66, as shown by line 216.

[0081] In an exemplary embodiment, identifying threatening behavior exhibited by individual 66 using system controller 34A at box 212 also includes receiving verbal communications from individual 66 using system controller 34A via external microphone 50, identifying threatening language using system controller 34A, capturing an image of individual 66 using a camera 38 that communicates with system controller 34A, and identifying at least one of threatening gestures made by individual 66 using computer vision algorithm 80 within system controller 34A.

[0082] In another exemplary embodiment, providing a warning to the individual 66 using the system controller 34A at box 206 also includes (moving to box 218) projecting a second graphic 68 onto a surface adjacent to the vehicle 10 using an external projector 54 in communication with the system controller 34A, the second graphic 68 being color coded and highlighting the location of the individual 66.

[0083] In an exemplary embodiment, providing a warning to the individual 66 using the system controller 34A at block 206 also includes determining using the system controller 34A after projecting the second graphic 68 at block 218 (moving to block 220) that the individual 66 is still within the safety zone 58, as indicated by line 222, and providing an audible warning to the individual 66 via the external speaker 52 (moving to block 224). If, at block 220, the individual 66 is still not within the safety zone 58, the method 200 returns to block 204, as indicated by line 226.

[0084] In another exemplary embodiment, the method 200 also includes (moving from block 224 to block 228 ) providing, using the system controller 34A, via the wireless communication module 36 , a warning signal to the personal device 74 carried by the individual 66 .

[0085] In an exemplary embodiment, providing a warning to the individual 66 using the system controller 34A at block 206 also includes determining using the system controller 34A that the individual 66 is still within the safety zone 58 after providing the audible warning to the individual 66 via the external speaker 52 at block 224 (moving to block 230), and providing a tactile warning signal using the system controller 34A to the personal device 74 carried by the individual 66 via the wireless communication module 36 (moving to block 232). If, at block 230, the system controller 34A determines that the individual 66 is still not within the safety zone 58, the method 200 returns to block 204, as indicated by line 226. After providing the tactile signal at block 232, the method 200 returns to block 204, as indicated by line 226.

[0086] In another exemplary embodiment, the method 200 further includes, at box 234, detecting when an occupant 60 in the vehicle 10 leaves the vehicle 10 using a plurality of onboard sensors 40a-40n, wherein, at box 202, projecting the first graphic 56 onto a surface adjacent to the vehicle 10 using an external projector 54 in communication with the system controller 34A also includes projecting the first graphic 56 onto a surface adjacent to the vehicle 10 and centered on the occupant 60 of the vehicle 10 using the external projector 54.

[0087] In an exemplary embodiment, providing a warning to a personal device 74 carried by an individual 66 using the system controller 34A via the wireless communication module 36 at box 228 also includes providing a warning message to the personal device 74 carried by the occupant 60 using the system controller 34A via the wireless communication module 36, and providing a tactile warning signal to the personal device 74 carried by the individual 66 via the wireless communication module 36 using the system controller 34A also includes providing a tactile warning signal to the personal device 76 carried by the occupant 60 via the wireless communication module 36 using the system controller 34A.

[0088] In another exemplary embodiment, the method 200 further includes receiving (moving to block 238) instructions from the occupant 60 via the wireless communication module 36 to provide a warning to the individual 66 within the projected safety zone 58 using the system controller 34A if the system controller 34A does not identify threatening behavior exhibited by the individual 66 at block 212, as indicated by line 236, wherein providing the warning to the individual 66 using the system controller 34A at block 206 also includes providing the warning to the individual 66 using the system controller 34A when the instruction is received from the occupant 60, as indicated by line 240. If, at block 238, no instruction is received from the occupant 60, the method 200 returns to block 204, as indicated by line 242.

[0089] An advantage of the disclosed system and method is that a warning is provided to an individual when the individual is within a safe zone.

[0090] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to be within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A method for providing a projected safety zone adjacent to a vehicle, comprising: projecting a first graphic onto a surface adjacent to the vehicle using an external projector in communication with a system controller, the first graphic defining a predetermined safety zone; detecting the presence of an individual located within the projected safety zone using a plurality of onboard sensors in communication with the system controller; as well as A warning is provided to the individual using the system controller.

2. The method according to claim 1, further comprising: determining, using the system controller, whether the individual has credentials authorizing the individual to be within the projection's secure zone; as well as Wherein, providing the warning to the individual using the system controller further comprises providing the warning to the individual using the system controller when the individual does not have a certificate authorizing the individual to be within the safe zone.

3. The method according to claim 2, wherein: Determining, using the system controller, whether the individual has credentials authorizing the individual to be within the projected safety zone further comprises: A digital certificate of the individual is accessed with the system controller via a wireless communication module, wherein the digital certificate is located in at least one of a cloud-based database and a personal device carried by the individual.

4. The method according to claim 2, wherein: Determining, using the system controller, whether the individual has credentials authorizing the individual to be within the projected safety zone further comprises: capturing an image of the individual using a camera in communication with the system controller; and A computer vision algorithm within the system controller is utilized to recognize a physical credential, wherein the physical credential includes at least one of a facial identifier, specific clothing, and a badge.

5. The method according to claim 2, further comprising: identifying, using the system controller, threatening behavior exhibited by the individual; as well as Wherein providing a warning to the individual using the system controller further comprises providing a warning to the individual using the system controller when the individual has a certificate authorizing the individual to be within the safe zone, and the system controller identifies threatening behavior exhibited by the individual.

6. The method according to claim 5, wherein: The identifying, using the system controller, threatening behavior exhibited by the individual further comprises at least one of: receiving, with the system controller, verbal communications from the individual via an external microphone, and identifying, with the system controller, threatening language; and An image of the individual is captured using a camera in communication with the system controller, and a threatening gesture made by the individual is identified using a computer vision algorithm within the system controller.

7. The method according to claim 2, wherein: Providing a warning to the individual using the system controller further includes projecting a second graphic onto the surface adjacent to the vehicle using the external projector in communication with the system controller, the second graphic being color coded and highlighting a location of the individual.

8. The method according to claim 7, wherein: Providing, using the system controller, a warning to the individual further comprises: After projecting the second graphic, determining, using the system controller, that the individual is still within the safety zone; and An audible warning is provided to the individual via an external speaker.

9. The method of claim 8, further comprising providing, using the system controller, a warning signal to a personal device carried by the individual via a wireless communication module.

10. The method according to claim 9, wherein: Providing, using the system controller, a warning to the individual further comprises: determining, using the system controller, that the individual is still within the safety zone after providing the audible warning to the individual via the external speaker; and A tactile warning signal is provided, using the system controller, via the wireless communication module to the personal device carried by the individual.

Images