Object detection system

Abstract

To provide an object detection system for a vehicle.SOLUTION: An object detection system for a vehicle includes a processor and a memory communicably coupled to the processor. The memory stores instructions that when executed by the processor cause the processor to receive a priority assignment set by an occupant of a vehicle. The priority assignment corresponds to a customization of at least one warning characteristic. The at least one warning characteristic includes a distance threshold. The instructions also cause the processor to detect an object in an external environment of the vehicle and apply the priority assignment to the object. The instructions further cause the processor to issue a warning according to the at least one warning characteristic. According to the distance threshold, the warning is issued when a distance from the vehicle to the object meets the distance threshold.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The embodiments disclosed in this specification relate to an object detection system for a vehicle, and more particularly, to an object detection system for a vehicle that can be customized by a vehicle occupant.

Background Art

[0002] Some vehicles include an object detection system that detects one or more objects near the vehicle and provides a warning to the vehicle occupant when the vehicle is close to an object. Such an object detection system may use various types of sensors for detecting objects, such as cameras, sonar sensors, lidar sensors, and / or radar sensors, and the various types of sensors can detect the presence of an object and determine the distance to the object. In some configurations, the object detection system can display a live view around the vehicle that includes the object, for example, a live camera view is displayed on the vehicle's user interface. Further, the object detection system can provide information to the vehicle occupant regarding the distance between the vehicle and the detected object.

Summary of the Invention

[0003] Embodiments of an object detection system for a vehicle and a method of operating an object detection system for a vehicle are disclosed herein.

[0004] In one embodiment, an object detection system is disclosed. The object detection system includes a processor and a memory communicably connected to the processor. The memory stores instructions, which, when executed by the processor, cause the processor to receive priority assignments set by the vehicle occupants. The priority assignments correspond to the customization of at least one warning characteristic, which includes a distance threshold. The instructions also cause the processor to detect objects in the external environment of the vehicle and apply the priority assignments to the objects. The instructions further cause the processor to issue a warning according to at least one warning characteristic, which is issued when the distance from the vehicle to an object meets the distance threshold.

[0005] In another embodiment, a method for operating an object detection system is disclosed. The method includes receiving a priority assignment set by the occupants of a vehicle. The priority assignment corresponds to the customization of at least one warning characteristic, which includes a distance threshold. The method also includes detecting an object in the external environment of the vehicle and applying the priority assignment to the object. The method further includes issuing a warning according to at least one warning characteristic. According to the distance threshold, a warning is issued if the distance from the vehicle to the object meets the distance threshold.

[0006] These and other embodiments are described in further detail below. [Brief explanation of the drawing]

[0007] Various features, advantages, and other applications of this embodiment will become clearer by referring to the following detailed description and drawings.

[0008] [Figure 1] This figure shows an example of a vehicle equipped with an object detection system. [Figure 2]This figure shows an example of a vehicle user interface that an occupant may use to customize one or more warning characteristics of an object detection system by assigning priorities to one or more classes of objects. [Figure 3A] This diagram illustrates an example where a vehicle backs up and exits a private road, and objects within the private road are detected by an object detection system. [Figure 3B] Figure 3A shows an example of a warning issued by the object detection system regarding an object detected in the figure. [Figure 4A] This figure illustrates an example where a vehicle is parked in a home garage, and objects within the garage are detected by an object detection system. [Figure 4B] This figure shows an example of assigning priority to objects detected in Figure 4A. [Figure 4C] Figure 4A shows an example of a warning issued by the object detection system regarding an object detected. [Figure 5] This figure shows an example of assigning priority to object locations. [Figure 6A] This figure shows an example of a monocular camera image captured by a monocular camera mounted on a vehicle. [Figure 6B] This figure shows an example of a depth map generated based on the monocular camera image in Figure 5A. [Figure 7] This figure shows an example of a monocular depth estimation system. [Figure 8] This figure shows an example of how to operate an object detection system. [Figure 9] This figure shows an example of how to predict object priority assignments based on historical data. [Figure 10] Figure 10 shows an example of how to detect one or more objects using a depth map. [Modes for carrying out the invention]

[0009] This disclosure describes an object detection system for a vehicle. The object detection system is customizable by the vehicle occupant by assigning priorities to object classes, specific objects, and / or object locations. The occupant can set the priority assignments by using the vehicle's user interface, and the priority assignments correspond to the customization of at least one warning characteristic, e.g., a distance threshold between the vehicle and the object. When an object is detected in the vehicle's external environment, the object detection system may classify the object and apply the priority assignments to the object. The object detection system may issue a warning to the occupant regarding the object according to a warning characteristic, e.g., a distance threshold. According to the distance threshold, a warning is issued when the distance between the vehicle and the object meets the distance threshold.

[0010] A typical passenger vehicle 100 is shown in Figure 1. In this explanation, the terms "forward," "towards the front," and similar terms, as well as "rearward," "towards the rear," and similar terms, refer to the longitudinal direction of vehicle 100. "Forward," "towards the front," and similar terms refer to the front (front part) of vehicle 100, while "rearward," "towards the rear," and similar terms refer to the rear (rear part) of vehicle 100. The terms "side," "lateral," "sideways," and similar terms refer to the lateral direction of vehicle 100, "driver's side" and similar terms refer to the left side of vehicle 100, and "passenger side" and similar terms refer to the right side of vehicle 100.

[0011] Vehicle 100 includes an external compartment and numerous internal compartments. The compartments include a passenger compartment and an engine compartment. In particular, vehicle 100 may include seats, a dash assembly, an instrument panel, a control unit, and similar components housed within the passenger compartment. Furthermore, vehicle 100 may include an engine, motors, a transmission, and similar components, as well as other powertrain components such as wheels, housed in the engine compartment and elsewhere within vehicle 100. The wheels support the rest of vehicle 100 on the ground. One, some, or all of the wheels are powered by the rest of the powertrain components to drive vehicle 100 along the ground.

[0012] Vehicle 100 includes one or more vehicle systems 102 that are operable to perform vehicle functions. In addition to the vehicle systems 102, vehicle 100 includes a sensor system 110, and one or more processors 120, memory 122, and control modules 124 that are communicatively connected to the vehicle systems 102 and the sensor system 110. The sensor system 110 is operable to detect information about vehicle 100. The processors 120, memory 122, and control modules 124 together function as one or more computing devices 118, the control modules 124 of which can be employed, whole or in part, to organize the operation of vehicle 100. Specifically, the control modules 124 operate the vehicle systems 102 based on information about vehicle 100. Thus, as a prerequisite for operating the vehicle systems 102, the control modules 124 collect information about vehicle 100, including information about vehicle 100 detected by the sensor systems 110. Next, the control module 124 evaluates information about the vehicle 100 and operates the vehicle system 102 based on that evaluation.

[0013] The vehicle system 102 is either part of the main body, attached to the main body, or otherwise supported by the main body. The vehicle system 102 may be housed, whole or in part, in any combination of the passenger compartment, engine compartment, or other locations in the vehicle 100. Each vehicle system 100 includes one or more vehicle elements. Representing the vehicle system 102 to which a vehicle element belongs, each vehicle element is operable to perform, whole or in part, any combination of the vehicle functions to which the vehicle system 102 is associated. It will be understood that the vehicle elements and the vehicle system 102 to which the vehicle elements belong may, but do not have to be, separate from each other.

[0014] The vehicle system 102 includes an energy system 104 and a propulsion system 106. The energy system 104 and the propulsion system 106 are connected to each other. Furthermore, the drivetrain is mechanically connected to the propulsion system 106. Together, the propulsion system 106 and the drivetrain function as the powertrain for the vehicle 100. The energy system 104 is operable to perform one or more energy functions, including but not limited to energy storage and, in other cases, energy processing. The propulsion system 106 is operable to perform one or more propulsion functions using energy from the energy system 104, including but not limited to powering the wheels.

[0015] The vehicle system 102 may also include a user interface 108. The user interface 108 may be any device, component, system, element, or mechanism, or a group thereof, that enables a user to input information / data into the machine. For example, the user interface 108 may be a touchscreen mounted on the center console of the vehicle 100. The user interface 108 may also be a user's mobile phone that is communicatively connected to the vehicle 100. The user interface 108 may receive input from the occupants of the vehicle 100, such as the driver or passengers. Information input to the user interface 108 may be stored in memory 122 by the processor 120. The user interface 108 may also output information to the users of the vehicle 100, such as the occupants.

[0016] As part of the sensor system 110, the vehicle 100 includes one or more vehicle sensors 112 and one or more environmental sensors 114. The vehicle sensors 112 monitor the vehicle 100 in real time. The vehicle sensors 112 are operable to detect information about the vehicle 100, including information about user requests and information about the operation of the vehicle 100, on behalf of the sensor system 110. For example, the vehicle sensors 112 may be configured to detect and / or acquire data about various operating parameters of the vehicle 100. For example, the vehicle sensors 112 may include one or more speedometers, one or more gyroscopes, one or more accelerometers, one or more inertial measuring units (IMUs), one or more wheel sensors, one or more steering angle sensors, one or more controller area network (CAN) sensors, and similar. In connection therewith, the sensor system 110 is operable to detect the position and movement of the vehicle 100, including information relating to the operation of the vehicle 100, such as the speed, acceleration, orientation, rotation, direction and similar, wheel movement, steering angle, and the operating state of one, some, or all of the vehicle systems 102.

[0017] The environmental sensor 114 can be configured to detect, determine, evaluate, monitor, measure, quantify, acquire, and / or sense data or information regarding the external environment in which the vehicle 100 is located or one or more portions thereof. The environmental sensor 114 can include one or more external cameras and one or more external sensors, such as temperature sensors, weather sensors, LIDAR, RADAR, and the like. The external cameras can include one or more monocular cameras 116. The environmental sensor 114 can be located outside the vehicle 100 or at any other suitable location in the vehicle 100. Using the environmental sensor 114, the vehicle system 102 can determine information regarding the external environment of the vehicle 100. For example, the vehicle system 102 can detect one or more objects in the external environment of the vehicle 100.

[0018] The vehicle system 102, the sensor system 110, the processor 120, the memory 122, and the control module 124 can be utilized to implement the object detection system 126. The vehicle system 102, the sensor system 110, the processor 120, the memory 122, and the control module 124, which are utilized to implement the object detection system 126, can be part of one or more other control systems typical of a vehicle in the vehicle 100 or can be dedicated to the object detection system 126.

[0019] The object detection system 126 can be configured to detect one or more objects near the vehicle 100 and issue a warning to the occupants of the vehicle 100 based on the distance between the vehicle 100 and the object. The occupants can be the driver of the vehicle 100 or another passenger in the vehicle 100. In some configurations, the object detection system 126 can display the detected objects in color while displaying the external environment of the vehicle 100 in grayscale. The object detection system 126 can issue a warning according to a set of default warning characteristics. The default warning characteristics can be the warning characteristics set by the manufacturer of the vehicle 100. The default warning characteristics can include a distance threshold between the vehicle 100 and the detected object. More specifically, the object detection system 126 can issue a warning when it detects an object near the vehicle 100 if the distance threshold is met. For example, the distance threshold can be 10 feet (about 3 m), and the object detection system 126 can issue a warning when it detects an object near the vehicle 100 when the vehicle 100 is at about 10 feet (about 3 m) or less from the object. The default warning characteristics can be stored in the memory 122 by the processor 120 and / or received by the processor 120 from the memory 122.

[0020] Warning characteristics may be customizable by the occupant in one or more ways. For example, the occupant may customize warning characteristics by assigning priorities to objects detected by the object detection system 126. The assignment of priorities corresponds to the customization of warning characteristics. Therefore, based on the assignment of priorities, the object detection system 126 may change one or more of the warning characteristics. The occupant can assign priorities via the user interface 108, and the assignment of priorities may be received by the processor 120. The assignment of priorities may include a range from low priority to high priority. The occupant may want to assign low priority to objects that will not cause significant damage to the vehicle 100 or pose a significant threat to the object in the event of a collision. For example, the occupant may want to assign low priority to objects such as tree branches, debris, curbs, and speed bumps. On the other hand, the occupant may want to assign high priority to objects that will cause significant damage to the vehicle 100 or pose a significant threat to the object in the event of a collision. For example, a crew member might want to assign a high priority to an object, such as a nearby vehicle, a wall and / or barricade, a pedestrian and / or cyclist, a shopping cart, or a tree.

[0021] The warning characteristics may include a distance threshold on which the warning is based, and therefore, the assignment of priority may correspond to the customization of the distance threshold. For example, as the priority increases from low priority to high priority, the distance threshold may also increase. More specifically, the assignment of low priority may correspond to a lower distance threshold, and the assignment of high priority may correspond to a higher distance threshold. Therefore, when the object detection system 126 detects a low-priority object, the object detection system 126 may be configured to issue a warning to the occupants when the vehicle 100 is relatively close to the object. On the other hand, when the object detection system 126 detects a high-priority object, the object detection system 126 may be configured to issue a warning to the occupants when the vehicle 100 is further away from the object.

[0022] In some mechanisms, the occupant may assign these priorities during the initial setup process of the vehicle 100. During initial setup, it may be advantageous to assign overall priorities based on one or more classifications of objects, and the object detection system 126 may be configured to detect one or more classifications of objects. For example, the occupant may assign different priorities to tree branches, nearby vehicles, debris, pedestrians, speed bumps, curbs, and any other object classifications. For example, the occupant may assign a low priority to all tree branches, a high priority to all nearby vehicles, a low priority to all debris, and a high priority to all pedestrians.

[0023] The occupant can use the user interface 108 in any preferred configuration to assign priorities. For example, the user interface 108 may display one or more classes of objects next to a slider icon. The occupant can use the slider icon to set priority assignments for object classes. For example, as shown in Figure 2, the user interface 108 displays a tree branch 200 and a first slider icon 202 for setting the priority of the tree branch 200. If the passenger sets the priority of the tree branch 200 to a low priority assignment, the object detection system 126 applies the low priority assignment to any other tree branches that are detected. More specifically, when the object detection system 126 detects an object, it classifies the object. If the object class is a tree branch, the object detection system 126 adapts the object to a low priority assignment and issues a warning about the tree branch based on the low priority assignment. Similarly, the user interface 108 also displays a pedestrian 204 and a second slider icon 206 for setting the priority of the pedestrian 204. If the occupant sets the priority of pedestrian 204 to a high priority assignment, the object detection system 126 applies the high priority assignment to other pedestrians that are detected. More specifically, when the object detection system 126 detects an object, it classifies the object. If the object's class is a pedestrian, the object detection system 126 adapts the object to a high priority assignment and issues a warning about the pedestrian based on the high priority assignment.

[0024] Other illustrative examples of the object detection system 126 are shown in Figures 3A–3B, 4A–4C, and 5. In these and other examples, the object detection system 126 may be advantageous when parking the vehicle 100. For example, the occupants may find the object detection system 126 particularly advantageous when parking the vehicle 100 near an object, such as a signpost, curb, or shrub, as will be described in more detail below, and even when parking the vehicle 100 in the garage of their own home. The object detection system 126 is also advantageous when the vehicle 100 is driving normally or backing up. For example, referring to Figure 3A, the vehicle 100 is shown backing out of a driveway 300. The object detection system 126 may detect an object 302 in the driveway 300, classify the object 302 as a tree branch 302, and determine that a low priority is assigned to the tree branch. As shown in Figure 3B, the object detection system 126 may issue a warning to the occupants when the vehicle 100 is relatively close to the tree branch 302. The warning may be a visual warning, such as a warning icon 304 in the user interface 108, as shown, and may also include an auditory warning and / or a tactile warning. In some examples, the warning may include displaying the tree branch 302 in color while displaying the environment in grayscale.

[0025] In some cases, the occupant may want to assign priority to specific objects. For example, referring to Figure 4A, the occupant may want to pay particular attention to an object in the home garage 400, such as a bicycle 402 stored in the home garage 400, located in front of the vehicle 100. Figure 4A shows the vehicle 100 parked in the home garage 400, and Figure 4B shows an overhead view of the vehicle 100 parked in the home garage 400 in the user interface 108. In some cases, the occupant may be able to select a specific object and set a priority for that object. For example, referring to Figure 4B, the occupant can select the bicycle 402, and the priority assignment may be done using the slider icon 404. Now, referring to Figure 4C, once a priority assignment has been set, the object detection system 126 may issue a warning to the occupant based on the specific priority assignment. For example, if the occupant designates the bicycle 402 as a high priority, the object detection system 126 may issue a warning when the vehicle 100 is relatively far from the bicycle 402. The warning may be a visual warning, such as a warning icon 406 in the user interface 108, as shown, and may also include an auditory warning and / or a tactile warning. In some examples, the warning may include displaying the bicycle 402 in color while displaying the environment in grayscale.

[0026] Therefore, in addition to, or as an alternative to, assigning priorities based on object classification, the occupant may assign priorities to each object. This can be advantageous for various objects that may not be classifiable by the object detection system 126. For example, the occupant may be parked in the garage of their home, where they store a china cabinet that cannot be classified by the object detection system 126. The occupant may assign a high priority to the china cabinet to ensure that the vehicle 100 does not approach it. Furthermore, in some examples, the occupant may want to assign priorities to specific object locations. Referring to Figure 5, the occupant may select a specific object location 500 in the user interface 108. Priority assignments can be set for the specific object location 500 using the slider icon 502.

[0027] As described above, in any of the examples above, the warning issued by the object detection system 126 may be a visual warning, such as a warning icon in the user interface 108, and may also include an auditory warning and / or a tactile warning. Therefore, the warning characteristics may include the warning type, warning intensity, and warning location, and thus the assignment of priority may correspond to the customization of the warning type, warning intensity, and warning location. The warning type may be an auditory warning (e.g., a beep), a visual warning (e.g., a flashlight or image displayed in the user interface 108), and / or a tactile warning (e.g., vibration through the steering wheel). With respect to auditory warnings, the warning intensity is the volume of the warning, for example, a quieter warning for a low-priority assignment and a louder warning for a high-priority assignment. With respect to visual warnings, the warning intensity is the brightness of the warning, for example, a dimmer flashlight for a low-priority assignment and a brighter flashlight for a high-priority assignment.

[0028] Alternatively, the warning intensity may be the size of the visual warning, for example, a smaller image for a low-priority assignment and a larger image for a high-priority assignment. With respect to tactile warnings, the warning intensity may be the strength of the vibration, for example, a weaker vibration for a low-priority assignment and a stronger vibration for a high-priority assignment. With respect to auditory warnings, the warning location may be a speaker located in the center of the vehicle 100 (e.g., connected to the user interface 108) or a speaker located on the driver's side of the vehicle 100. With respect to visual warnings, the warning location may be a screen behind the steering wheel, a screen located in the center of the vehicle 100 (e.g., the user interface 108), a head-up display on the windshield, etc. With respect to tactile warnings, the warning location may be the steering wheel, seat, etc.

[0029] In some cases, the object detection system 126 may be able to predict the assignment of object priorities based on historical data regarding past priority assignments by the occupant. The historical data may be a database of past priority assignments made by the occupant. For example, if the historical data indicates that the occupant has designated all tree branches as low priority, the object detection system 126 may be able to infer that the occupant does not care about driving over tree branches and therefore may designate future detected tree branches as low priority. On the other hand, if the historical data indicates that the occupant has designated small tree branches as low priority and large tree branches as high priority, the object detection system 126 may be able to infer that the occupant does not want to drive over larger tree branches (for example, if the occupant does not want to damage their car) and therefore may designate future detected larger tree branches as high priority.

[0030] In another example, the object detection system 126 may learn the occupant's preferences based on the direction the vehicle 100 is traveling. For example, in some cases, the occupant may not want early warnings about objects in front of the vehicle 100 when driving forward, and the object detection system 126 may designate objects in front of the vehicle 100 as low priority when driving forward. On the other hand, if the occupant wants early warnings when driving in reverse, the object detection system 126 may designate objects behind the vehicle 100 as high priority when driving in reverse. If the object detection system 126 correctly predicts the priority assignment, it may add the priority assignment to a database, which may be stored in memory 122.

[0031] In some mechanisms, the object detection system 126 may detect one or more objects using a depth map generated based on a monocular camera image. Therefore, referring back to Figure 1, the vehicle system 102, sensor system 110, processor 120, memory 122, and control module 124 may be utilized to implement a monocular depth estimation (MDE) system 128. The vehicle system 102, sensor system 110, processor 120, memory 122, and control module 124, utilized to implement the MDE system 128, may be part of one or more other control systems typical of the vehicle in the vehicle 100, or they may be dedicated to the MDE system 128. The MDE system 128 is described in more detail below in relation to the monocular camera image 600 in Figure 6A and the depth map 602 in Figure 6B. The MDE system 128 may be configured to generate a depth map 602 of at least a portion of the external environment of the vehicle 100 based on information received from the sensor system 110. More specifically, the MDE system 128 may be configured, at least in part, to generate a depth map 602 based on information received by one or more monocular cameras 116 mounted on the vehicle 100. The depth map 602 can then be used as input to other vehicle systems 102, such as the object detection system 126, as will be described in more detail below.

[0032] As described above, the vehicle 100 may include one or more monocular cameras 116 mounted on the exterior of the vehicle 100 at the front or rear of the vehicle 100, inside the vehicle 100, and / or at any other location on the vehicle 100. The monocular cameras 116 may be configured to capture one or more monocular camera images 600 of the external environment of the vehicle 100. Referring here to Figure 6A, an example of a monocular camera image 600 is shown. The monocular camera image 600 shows a figure of a preceding vehicle traveling immediately in front of the vehicle 100. The monocular camera image 600 may be a color image typical of other types of on-board cameras. Referring here to Figure 6B, an example of a depth map 602 is shown. The depth map 602 is a monochrome image based on the monocular camera image 600. The pixel values ​​of the depth map 602 are proportional to the distance between the monocular camera 116 and the object in the monocular camera image 600. As shown, the pixel values ​​of the depth map 602 are proportional to the distance between the monocular camera 116 and the preceding vehicle 100. The object detection system 126 can use the depth map 602 to identify objects near the vehicle 100.

[0033] Referring here to Figure 7, the MDE system 128 is shown. The MDE system 128 may be configured to receive an input 700 and generate an output 750. The input 700 may be a monocular camera image 600. The monocular camera image 600 may be a color image taken by a monocular camera 116. The output 750 may be identification information of one or more objects, for example, one or more objects in the external environment of the vehicle 100. The MDE system 128 includes a monocular depth estimation (MDE) module 710, a road segmentation extraction (RSE) module 720, a feature extraction module 730, and one or more processors 740. The processor 740 may be the processor 120 in Figure 1 or any other suitable processor. The MDE module 710, the RSE module 720, and / or the feature extraction module 730 may be components of the processor 740 or components of one or more other processors. The MDE module 710 is configured to receive input 700 (i.e., monocular camera image 600) and generate a depth map 602 using machine learning or any other preferred method. As described above, the depth map 602 is a grayscale image in which each pixel value is proportional to the distance to the monocular camera 116. The RSE module 720 is configured to receive the monocular camera image 600 and / or depth map 602 and detect, segment, and extract portions of the monocular camera image 600 and / or depth map 602 corresponding to the road. The feature extraction module 730 may receive input 700 and may be configured to detect features (e.g., objects) within the monocular camera image 600. The processor 740 may function as a decision system based on input 700 (i.e., monocular camera image 600), depth map 602, the road, and / or features to generate an output 750. The output 750 may be the detection of one or more objects in the external environment of the vehicle 100.

[0034] Referring again to Figure 1, as described above, the processor 120, memory 122, and control module 124 together function as a computing device 118, the control module 124 organizing the operation of the vehicle 100, including but not limited to the operation of the vehicle system 102. The control module 124 may be a dedicated control module for the object detection system 126 and / or the MDE system 128. In connection therewith, as part of a central control system, the vehicle 100 may include a global control unit (GCU) to which the control module 124 is communicably connected. Alternatively, the control module 124 may be a global control module. In connection therewith, as part of a central control system, the vehicle 100 may include a global control unit (GCU) to which the control module 124 belongs. Although the shown vehicle 100 includes one control module 124, it will be understood that this disclosure is in principle applicable to other similar vehicles that include multiple control modules. Furthermore, although the control module 124 is shown as part of the vehicle 100, it will be understood that the control module 124 may be located outside the vehicle 100.

[0035] The processor 120 may be any component configured to execute any of the processes described herein, or any form of instructions that execute such processes or cause such processes to execute. The processor 120 may be implemented using one or more general-purpose or dedicated processors. Examples of suitable processors include microprocessors, microcontrollers, digital signal processors, or other forms of circuitry that execute software. Other examples of suitable processors include, but are not limited to, central processing units (CPUs), array processors, vector processors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), application-specific integrated circuits (ASICs), programmable logic circuits, or controllers.

[0036] The processor 120 may include at least one hardware circuit (e.g., an integrated circuit) configured to execute instructions contained within the program code. In a mechanism with multiple processors, the processors may function independently of each other or in combination. Furthermore, although the processor 120 is shown as part of the vehicle 100, it will be understood that the processor 120 may be located outside the vehicle 100. The memory 122 is a non-temporary computer-readable medium. The memory 122 may include volatile or non-volatile memory, or both.

[0037] Examples of suitable memory include random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. Memory 122 includes instructions stored in program code. Such instructions are executable by the processor 120 or the control module 124. Memory 122 may be part of the processor 120 or the control module 124, or may be communicatively connected to the processor 120 or the control module 124. Generally speaking, the control module 124 includes instructions that can be executed by the processor 120. When executed by the processor 120, the control module 124 may be implemented as computer-readable program code that performs one or more of the processes described herein. Such computer-readable program code may be stored in memory 122. The control module 124 may be part of the processor 120, or it may be communicatively connected to the processor 120.

[0038] Having described various possible systems, devices, elements, and / or components, we now describe various methods, including various possible steps of the method. While the methods described may be applicable to the mechanisms described above, it should be understood that such methods may be performed on other suitable systems and mechanisms. Such methods may include other steps not shown herein, and are not limited to including all steps shown. The blocks shown herein as part of the method are not limited to a specific chronological order. In fact, some blocks may be performed in a different order than shown, and / or at least some of the blocks shown may occur simultaneously.

[0039] Referring to Figure 8, an example of operating the object detection system 126 is shown. Method 800 may begin in operation 810. In operation 820, method 800 may include receiving a priority assignment set by the occupants of vehicle 100. The priority assignment corresponds to the customization of at least one warning characteristic, which includes a distance threshold. In operation 830, method 800 includes detecting an object in the external environment of vehicle 100. In operation 840, method 800 includes applying the priority assignment to the object. In operation 850, method 800 includes issuing a warning according to the warning characteristic. A warning is issued if the distance from vehicle 100 to the object meets the distance threshold. Method 800 may end in operation 860.

[0040] Referring to Figure 9, an example of method 900 for predicting priority assignments for objects detected near vehicle 100 is shown. Method 900 begins in operation 910. In operation 920, method 900 includes receiving historical data that includes input relating the types of objects detected near vehicle 100 to occupant selection priorities for the objects. In operation 930, method 900 includes detecting one or more objects in the external environment of vehicle 100. In operation 940, method 900 includes predicting a priority assignment for at least one of the objects based on the historical data. Method 900 may end in operation 950.

[0041] Referring to Figure 10, an example of method 1000 for detecting one or more objects is shown. Method 1000 may begin in operation 1010. In operation 1020, method 1000 may include receiving a monocular camera image 600 from the vehicle 100's sensor system 110. In operation 1030, method 1000 may include generating a depth map 602 based on the monocular camera image 600. The depth map 602 may be a grayscale image in which each pixel value is proportional to the distance to the monocular camera 116. In operation 1040, method 1000 may include detecting, segmenting, and extracting a portion of the depth map 602 corresponding to the plane on which the vehicle 100 is traveling, based on the monocular camera image 600 and / or the depth map 602. In operation 1050, method 1000 may include detecting and extracting features based on the monocular camera image 600 and / or the depth map 602. For example, method 1000 may include detecting one or more objects based on a monocular camera image 600 and / or a depth map 602. In operation 1060, method 1000 may include detecting one or more objects in the external environment of the vehicle 100.

[0042] While the enumerated characteristics and conditions of the present invention are described in relation to specific embodiments, it should be understood that the present invention is not limited to the disclosed embodiments, but rather is intended to include various modifications and equivalent mechanisms that fall within the spirit and scope of the appended claims, the scope of which should be given the broadest possible interpretation to encompass all such modifications and equivalent structures as permitted under the law. The inventions disclosed herein include the following embodiments: [Aspect 1] An object detection system, Processor and A memory that is communicably connected to the processor and stores instructions, wherein when an instruction is executed by the processor, the processor receives the instructions. The system receives priority assignments set by the vehicle occupants, the priority assignments correspond to the customization of at least one warning characteristic, and the at least one warning characteristic includes a distance threshold. To detect objects in the external environment of the vehicle, Apply the aforementioned priority assignment to the object, A memory that issues a warning according to at least one warning characteristic, and issues the warning when the distance from the vehicle to the object satisfies the distance threshold, An object detection system equipped with the following features. [Aspect 2] The object detection system according to Embodiment 1, wherein the priority assignment is set by the crew for an object classification, and the instruction further causes the processor to apply the priority assignment to the object when the object falls within the object classification. [Aspect 3] The object detection system according to embodiment 2, wherein the object classification is at least one of trees, tree branches, speed bumps, debris, curbs, walls, barricades, vehicles, pedestrians, people riding bicycles, and shopping carts. [Aspect 4] The object detection system according to Embodiment 1, wherein the priority assignment is set by the crew for a specific object, and the instruction further causes the processor to apply the priority assignment to the object if the object is the specific object. [Aspect 5] The object detection system according to Embodiment 1, wherein the priority assignment is set by the crew for an object location, and the instruction further causes the processor to apply the priority assignment to the object when the object is at the object location. [Aspect 6] The object detection system according to Embodiment 1, wherein the priority assignment is a priority assignment within a range of priority assignments from low priority to high priority, and the distance threshold increases as the priority assignment increases from low priority to high priority. [Aspect 7] The object detection system according to Embodiment 1, wherein the instruction further causes the processor to create a database of historical data having inputs relating object classification to priority assignment, and the instruction further causes the processor to predict the priority assignment of detected objects based on the historical data. [Aspect 8] The object detection system according to embodiment 1, wherein the at least one warning characteristic includes at least one of a warning type, a warning intensity, and a warning location. [Aspect 9] The object detection system according to embodiment 1, wherein the instruction further includes causing the processor to display the external environment of the vehicle in grayscale and to issue the warning, and to display the object in color. [Aspect 10] The above instruction further instructs the processor to: The object is detected using at least one monocular camera image acquired by a monocular camera mounted on the vehicle. The object detection system according to embodiment 1, wherein the distance from the vehicle to the object is determined using a depth map generated based on the at least one monocular camera image. [Aspect 11] A method for operating an object detection system, wherein the method is Receiving a priority assignment set by the vehicle occupants, wherein the priority assignment corresponds to the customization of at least one warning characteristic, and the at least one warning characteristic includes a distance threshold, The detection of objects in the external environment of the vehicle, Applying the aforementioned priority assignment to the object, The means of issuing a warning according to at least one warning characteristic, wherein the warning is issued when the distance from the vehicle to the object satisfies the distance threshold, Methods that include... [Aspect 12] The method according to aspect 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for an object classification, and applying the priority assignment to the object includes applying the priority assignment to the object if the object falls within the object classification. [Aspect 13] The method according to embodiment 12, wherein the object classification is at least one of trees, tree branches, speed bumps, debris, curbs, walls, barricades, vehicles, pedestrians, cyclists, and shopping carts. [Aspect 14] The method according to aspect 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for a particular object, and applying the priority assignment to the object includes applying the priority assignment to the object if the object is the particular object. [Aspect 15] The method according to embodiment 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for an object location, and applying the priority assignment to the object includes applying the priority assignment to the object if the object is at the object location. [Aspect 16] The method according to embodiment 11, wherein the priority assignment is a priority assignment within a range of priority assignments from low priority to high priority, and the distance threshold increases as the priority assignment increases from low priority to high priority. [Aspect 17] To create a database of historical data with inputs that associate object classification with priority assignment, To predict the assignment of the priority of the detected object based on the historical data, The method according to embodiment 11, further comprising: [Aspect 18] The method according to embodiment 11, wherein the at least one warning characteristic includes at least one of a warning type, a warning intensity, and a warning location. [Aspect 19] The method according to embodiment 11, further comprising displaying the external environment of the vehicle in grayscale, and issuing the warning comprising displaying the object in color. [Aspect 20] The object is detected using at least one monocular camera image acquired by a monocular camera mounted on the vehicle, The distance from the vehicle to the object is determined using a depth map generated based on the at least one monocular camera image, The method according to embodiment 11, further comprising:

Claims

1. An object detection system, Processor and A memory that is communicably connected to the processor and stores instructions, wherein when an instruction is executed by the processor, the processor receives the instructions. The system receives priority assignments set by the vehicle occupants, the priority assignments correspond to the customization of at least one warning characteristic, and the at least one warning characteristic includes a distance threshold. To detect objects in the external environment of the vehicle, To classify the object for object classification and identification, Based on the object classification, the priority assignment is applied to the object. A memory that issues a warning according to at least one warning characteristic, and issues the warning when the distance from the vehicle to the object satisfies the distance threshold, An object detection system equipped with the following features.

2. The object detection system according to claim 1, wherein the priority assignment is set by the crew for the object classification, and the instruction further causes the processor to apply the priority assignment to the object when the object falls within the object classification.

3. The object detection system according to claim 2, wherein the object classification is at least one of trees, tree branches, speed bumps, debris, curbs, walls, barricades, vehicles, pedestrians, people riding bicycles, and shopping carts.

4. The object detection system according to claim 1, wherein the priority assignment is set by the crew for a specific object, and the instruction further causes the processor to apply the priority assignment to the object if the object is the specific object.

5. The object detection system according to claim 1, wherein the priority assignment is set by the crew for an object location, and the instruction further causes the processor to apply the priority assignment to the object when the object is at the object location.

6. The object detection system according to claim 1, wherein the priority assignment is a priority assignment within a range of priority assignments from low priority to high priority, and the distance threshold increases as the priority assignment increases from low priority to high priority.

7. The object detection system according to claim 1, wherein the instruction further causes the processor to create a database of historical data having inputs relating the object classification to priority assignments, and the instruction further causes the processor to predict the priority assignment of detected objects based on the historical data.

8. The object detection system according to claim 1, wherein the at least one warning characteristic includes at least one of a warning type, a warning intensity, and a warning location.

9. The object detection system according to claim 1, wherein the instruction further includes causing the processor to display the external environment of the vehicle in grayscale and to issue the warning, and to display the object in color.

10. The above instruction further instructs the processor to: The object is detected using at least one monocular camera image acquired by a monocular camera mounted on the vehicle. The object detection system according to claim 1, wherein the distance from the vehicle to the object is determined using a depth map generated based on the at least one monocular camera image.

11. A method for operating an object detection system, wherein the method is Receiving a priority assignment set by the vehicle occupants, wherein the priority assignment corresponds to the customization of at least one warning characteristic, and the at least one warning characteristic includes a distance threshold, The detection of objects in the external environment of the vehicle, To classify and identify the object, the object is classified, Applying the priority assignment to the object based on the object classification, The method involves issuing a warning according to at least one warning characteristic, wherein the warning is issued when the distance from the vehicle to the object satisfies the distance threshold, Methods that include...

12. The method according to claim 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for the object classification, and applying the priority assignment to the object includes applying the priority assignment to the object if the object falls within the object classification.

13. The method according to claim 12, wherein the object classification is at least one of trees, tree branches, speed bumps, debris, curbs, walls, barricades, vehicles, pedestrians, cyclists, and shopping carts.

14. The method according to claim 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for a particular object, and applying the priority assignment to the object includes applying the priority assignment to the object if the object is the particular object.

15. The method according to claim 11, wherein receiving a priority assignment set by a vehicle occupant includes receiving a priority assignment set by the occupant for an object location, and applying the priority assignment to the object includes applying the priority assignment to the object if the object is at the object location.

16. The method according to claim 11, wherein the priority assignment is a priority assignment within a range of priority assignments from low priority to high priority, and the distance threshold increases as the priority assignment increases from low priority to high priority.

17. Creating a database of historical data having inputs that associate the object classification with priority assignment, To predict the assignment of the priority of the detected object based on the historical data, The method according to claim 11, further comprising:

18. The method according to claim 11, wherein the at least one warning characteristic includes at least one of a warning type, a warning intensity, and a warning location.

19. The method according to claim 11, further comprising displaying the external environment of the vehicle in grayscale, and the issuing of the warning comprising displaying the object in color.

20. The object is detected using at least one monocular camera image acquired by a monocular camera mounted on the vehicle, The distance from the vehicle to the object is determined using a depth map generated based on the at least one monocular camera image, The method according to claim 11, further comprising:

Images