Method and apparatus for object identification and search scope adjustment

Abstract

To effectively identify an object of interest to an occupant of a vehicle.SOLUTION: The method includes processing a query to identify an object external to the vehicle, the query being made by an occupant of the vehicle. The method also includes determining a focus direction of the occupant when the query is received. The method further includes detecting a number of lanes of a road on which the vehicle is traveling. In addition, the method includes setting, in response to the query, one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and the side of the road, wherein the side of the road is located forward in the focus direction of the occupant. The method also includes setting one or more geo-fences corresponding to the one or more search areas for the vehicle on a geographic map in response to the query.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] Device and vehicle manufacturers are constantly challenged to provide products and services that provide value and benefits to users, such as vehicle occupants. Vehicle occupants often see objects or points of interest from their vehicles while driving and inquire about what these objects or points of interest are. Some vehicles have infotainment or navigation systems that utilize on-board cameras and / or sensors to capture gestures made by the vehicle occupant and / or track the eye movements of the vehicle occupant. Summary of the Invention

[0002] An aspect of the present disclosure relates to a method. The method includes processing a query to identify an object outside the vehicle, the query being made by an occupant of the vehicle. The method also includes determining a focus direction of the occupant when the query is received. The method further includes detecting a number of lanes on a road on which the vehicle is traveling. In addition, the method includes, in response to the query, setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. The method also includes, in response to the query, setting one or more geographic fences on a geographic map corresponding to the one or more search areas for the vehicle. The method further includes searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. In addition, the method includes searching a database for information about candidate objects of interest within the one or more geographic fences. The method also includes, in response to the query, generating an output including information about the candidate objects of interest.

[0003] An aspect of the present disclosure relates to an apparatus. The apparatus includes a processor and a memory, the memory having instructions stored therein that, when executed by the processor, cause the apparatus to process a query to identify an object outside the vehicle, the query being made by an occupant of the vehicle. The apparatus also performs the step of determining a focus direction of the occupant when the query is received. The apparatus further performs the step of detecting the number of lanes of a road on which the vehicle is traveling. Additionally, in response to the query, the apparatus performs the step of setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. In response to the query, the apparatus also performs the step of setting one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas. The apparatus further performs the step of searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. Additionally, the apparatus performs the step of searching a database for information about candidate objects of interest within the one or more geographic fences. In response to the query, the apparatus also performs the step of generating an output including information about the candidate objects of interest.

[0004] An aspect of the present disclosure relates to a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor, cause an apparatus to process a query for identifying an object outside the vehicle, the query being made by an occupant of the vehicle. The apparatus also performs the step of determining a focus direction of the occupant when the query is received. The apparatus further performs the step of detecting the number of lanes of a road on which the vehicle is traveling. Additionally, in response to the query, the apparatus performs the step of setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. In response to the query, the apparatus also performs the step of setting one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas. The apparatus further performs the step of searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. Additionally, the apparatus performs the step of searching a database for information about candidate objects of interest within the one or more geographic fences. In response to the query, the apparatus also performs the step of generating an output including information about the candidate objects of interest. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a flowchart of a method for adjusting a search scope to identify objects of interest to a vehicle occupant, according to one or more embodiments. [Figure 2] FIG. 2 is a perspective view of a vehicle according to one or more embodiments. [Figure 3] FIG. 3 is a graphical user interface according to one or more embodiments. [Figure 4] FIG. 4 is a graphical user interface according to one or more embodiments. [Figure 5] FIG. 5 is a block diagram of a system for adjusting a search scope to identify objects of interest to a vehicle occupant, according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0006] Aspects of the present disclosure are best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that, in accordance with standard practice in the industry, various features have not been drawn to scale. In fact, the dimensions of various features may be arbitrarily increased or decreased for clarity of discussion.

[0007] The following disclosure provides many different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components, values, operations, materials, arrangements, or equivalents thereof are described below to simplify the disclosure. It should be understood that these are merely examples and are not intended to be limiting. Other components, values, operations, materials, arrangements, or equivalents thereof are contemplated. For example, the formation of a first feature above or on a second feature in the following description includes embodiments in which the first and second features are formed in direct contact with each other, as well as embodiments in which an additional feature is formed between the first and second features such that the first and second features are not in direct contact with each other. Additionally, the present disclosure may repeat reference numerals and / or letters in various examples. This repetition is for purposes of simplicity and clarity and does not, in itself, dictate a relationship between the various embodiments and / or configurations discussed.

[0008] Additionally, spatially relative terms such as "below," "below," "below," "above," "on top," and the like may be used herein to describe the relationship of one element or feature to another element or feature when shown in the figures for ease of description. Spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors may likewise be interpreted accordingly.

[0009] Vehicle occupants often see objects or points of interest from their vehicles while driving and inquire about what these objects or points of interest are. Some vehicles have infotainment or navigation systems that can search for addresses associated with points of interest based on user input. Some infotainment or navigation systems have one or more displays and navigation capabilities and receive user input from vehicle occupants via a touchscreen, microphone, buttons, knobs, joystick, trackpad, motion sensors, cameras, or other suitable controllers.

[0010] However, problems arise for infotainment or navigation systems that provide responses to vehicle occupant queries regarding objects and / or points of interest because the object or point of interest identification system may fail to detect and / or identify the object or point of interest on the map if the query relates to an object or point of interest that is outside the search area or search range. This failure may occur because the number of lanes between the side of the vehicle and the side of the road on which the vehicle is traveling results in a search range that is not large enough to reach the object beyond the side of the road. For example, if there are more lanes on the right side of the vehicle than on the left side of the vehicle, the object will be located relatively farther from the vehicle or driver if it is on the right side of the vehicle than if it is on the left side of the vehicle.

[0011] The present disclosure includes methods and systems for adjusting a search scope to identify objects of interest to vehicle occupants.

[0012] FIG. 1 is a flowchart of a method 100 for adjusting a search scope to identify objects of interest to a vehicle occupant, according to one or more embodiments.

[0013] In some embodiments, method 100 is implemented using system 500 (FIG. 5). In some embodiments, method 100 is implemented using a system other than system 500 (FIG. 5).

[0014] In some embodiments, method 100 is implemented in vehicle 200 (FIG. 2). In some embodiments, method 100 is implemented in a vehicle other than vehicle 200 (FIG. 2).

[0015] In some embodiments, method 100 includes detecting a query about an object of interest that a vehicle occupant is viewing based on user input, such as a detected pointing gesture, gaze direction, verbal query, or some other suitable manner in which a vehicle occupant may interact with the vehicle's infotainment system or navigation system.

[0016] In operation 101, a query to identify an object outside the vehicle is processed. The query is made by an occupant of the vehicle. In some embodiments, the query is based at least in part on a detected pointing gesture by the occupant of the vehicle. In some embodiments, a gaze direction of the occupant of the vehicle is detected. In some embodiments, the query is based on verbal commands or words spoken out loud by the occupant of the vehicle. In some embodiments, one or more of the direction of the pointing gesture or the gaze direction is the focus direction of the occupant when the query is received. In some embodiments, the occupant's focus direction is based solely on a detected pointing gesture made by the occupant. In some embodiments, the occupant's focus direction is based solely on a detected line of sight of the occupant. In some embodiments, the occupant's focus direction is based both on a detected pointing gesture made by the occupant and on a detected line of sight of the occupant.

[0017] In some embodiments, the vehicle occupant is the driver of the vehicle. In some embodiments, the vehicle occupant is a passenger of the vehicle. In some embodiments, the driver of the vehicle sits behind the steering wheel of the vehicle. In some embodiments, the vehicle passenger sits in a seat other than the driver's seat, such as a front passenger seat or a rear passenger seat. In some embodiments, the vehicle occupant is any passenger of the vehicle who is sitting or standing in the vehicle.

[0018] In some embodiments, one or more of the pointing motion or gaze direction are detected by one or more sensors or cameras in the vehicle. In some embodiments, the vehicle sensor or camera detecting the pointing motion and / or gaze direction of the vehicle occupant is an interior sensor or camera, such as a sensor or camera facing the interior of the vehicle. In some embodiments, the vehicle sensor or camera detecting the pointing motion and / or gaze direction of the vehicle occupant is an outward-facing sensor or camera, such as a sensor or camera located inside the vehicle cabin and facing outward from the cabin. In some embodiments, the interior and exterior vehicle sensors or cameras detecting the pointing motion and / or gaze direction of the vehicle occupant are physically located inside the vehicle. In some embodiments, the interior vehicle sensor or camera detecting the pointing motion and / or gaze direction of the vehicle occupant is located on the exterior of the vehicle facing the interior of the vehicle. In some embodiments, the exterior vehicle sensor or camera detecting the pointing motion and / or gaze direction of the vehicle occupant is located on the exterior of the vehicle facing outward from the cabin. In some embodiments, the pointing and / or gaze direction is detected by a combination of an internal vehicle sensor or camera that detects the pointing and / or gaze direction of the vehicle occupant and an external vehicle sensor or camera that detects the pointing and / or gaze direction of the vehicle occupant.

[0019] In some embodiments, verbal commands or words spoken by a vehicle occupant are included in the context data, which is audio data received by a microphone associated with the vehicle, within a predetermined time period that includes the time the pointing gesture and / or gaze direction is detected. In some embodiments, the microphone is an interior microphone configured to capture sounds within the vehicle cabin. In some embodiments, the microphone is an exterior microphone configured to capture sounds outside the vehicle. In some embodiments, the microphone is configured to capture audio commands as user input. In some embodiments, the context data includes one or more of a conversation between two or more vehicle occupants, a conversation between a vehicle occupant and a person outside the vehicle, such as via a telephone, video call, or other suitable form of communication, a conversation between a vehicle occupant and a person outside the vehicle, such as through a window or through the vehicle's exterior speakers, a verbal inquiry made by a vehicle occupant, music, other sounds, externally captured sounds, or other suitable interior or exterior sounds that can be captured by one or more microphones associated with the vehicle.

[0020] In operation 103, the number of lanes on the road the vehicle is traveling on is detected. For example, the road may have one lane, two oncoming lanes, three lanes on each side with one or more oncoming lanes, or no oncoming lanes.

[0021] In operation 105, it is detected which lane out of the number of lanes the vehicle is traveling in.

[0022] In operation 107, one or more search areas are established extending from the vehicle in response to the query. In some embodiments, the one or more search areas are established based on a determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in a focus direction of the occupant. In some embodiments, the one or more search areas are established based on a determined number of lanes between the vehicle and a first side of the road and a second side of the road opposite the first side of the road. In some embodiments, establishing the one or more search areas is further based on a lane in which the vehicle is traveling.

[0023] In some embodiments, the first search area extends in a first direction from the vehicle and the second search area extends in a second direction from the vehicle that is different from the first direction. In some embodiments, the first search area extends toward a first side of the road and the second search area extends toward a second side of the road.

[0024] The second search area is set to be larger than the first search area based on the number of lanes between the vehicle and the first side of the road being less than the number of lanes between the vehicle and the second side of the road, and based on the first side of the road being closer to the vehicle than the second side of the road.

[0025] The first search area is set to be larger than the second search area based on the number of lanes between the vehicle and the first side of the road being greater than the number of lanes between the vehicle and the second side of the road, and based on the first side of the road being farther from the vehicle than the second side of the road.

[0026] The first search area is set to be equal to the second search area based on the number of lanes between the vehicle and the first side of the road being equal to the number of lanes between the vehicle and the second side of the road, and based on the first side of the road and the second side of the road being equidistant from the vehicle.

[0027] In some embodiments, the first search area extends beyond a first side of the road by a first distance, and the second search area extends beyond a second side of the road by a second distance, the first distance being equal to the second distance.

[0028] For example, if a vehicle is traveling in the center lane of a three-lane road with no oncoming traffic, there is one lane on each side of the vehicle. If a vehicle is traveling in the left lane of a three-lane road with no oncoming traffic, there are two lanes to the right of the vehicle. If a vehicle is traveling in the right lane of a three-lane road with no oncoming traffic, there are two lanes to the left of the vehicle. Other combinations of lane numbers and traffic patterns can be identified and quantified to establish the first and / or second search areas, including oncoming lanes, lanes over the median, turn lanes, on-ramps, off-ramps, etc.

[0029] In some embodiments, the one or more search areas are each a triangle with each vertex of the triangle located at the vehicle.

[0030] In some embodiments, the one or more search areas adapt to vehicle speed and increase in size from a first set size based on an increase in vehicle speed. In some embodiments, the one or more search areas adapt to vehicle lane changes and increase or decrease in size accordingly from a first set size based on detected lane changes.

[0031] In some embodiments, one or more search areas are reduced in size based on the detected lane change and the lane the vehicle is traveling in. For example, if the vehicle moves into the right lane and the vehicle occupant points to the right, the size of the search area toward the right side of the vehicle is reduced.

[0032] Similarly, if the vehicle moves into the right lane and the vehicle occupant points to the left, the size of the search area toward the left side of the vehicle is increased.

[0033] In some embodiments, the size of one or more search areas is gradually increased until at least one candidate object of interest is identified in map data searched for information regarding the candidate object of interest. In some embodiments, contextual information is processed as a component for determining how large the search area should be and / or for determining when incremental increases in the search area will capture potential candidate object of interest. For example, if a vehicle occupant points to a distant mountain and talks about it, even though there is a store in the search area between the vehicle and the mountain, the system recognizes that the vehicle occupant is inquiring about the mountain and expands the search area to include the distant mountain.

[0034] In some embodiments, if the system is unable to identify a potential object of interest or information corresponding to a potential object of interest based on the context information, the system incrementally increases the size of the search area up to a maximum size until a potential object of interest candidate is identified.

[0035] In some embodiments, setting the one or more set areas includes adjusting the size of at least one of the one or more search areas based on one or more of a detected lane change, a change in occupant focus direction, vehicle speed, a change in vehicle speed, or some other suitable criteria for increasing or decreasing the size of the search area.

[0036] In operation 109, in response to the query, one or more geographic fences corresponding to one or more search areas are established for the vehicle on the geographic map.

[0037] In some embodiments, the first search area and the second search area are all contained within a search area corresponding to one or more geographic fences for identifying candidate objects of interest.

[0038] In some embodiments, the first search area and the second search area are all included within a search area corresponding to one or more geographic fences for identifying potential objects of interest and one or more additional potential objects of interest around the vehicle at the time of the query.

[0039] In some embodiments, setting one or more search areas is performed to include one of the first search area or the second search area based on the occupant's focus direction, and the system refrains from including one of the first search area or the second search area based on the occupant's focus direction.

[0040] In operation 111, map data associated with the geographic map is searched to identify candidate objects of interest within one or more geographic fences.

[0041] In operation 113, a database is searched for information about candidate objects of interest within one or more geographic fences.

[0042] In operation 115, in response to the query, an output is generated that includes information about the object candidates.

[0043] In some embodiments, the output is a graphical user interface output by a display of the vehicle or a display of a mobile device communicatively coupled to the vehicle, hi some embodiments, the display is associated with the vehicle's infotainment system or navigation system.

[0044] In some embodiments, the graphical user interface includes a geographic map, a location icon indicating a location on the geographic map where a query was made, graphical objects representing one or more search areas or one or more geographic fences, and descriptive information associated with the candidate objects of interest, the graphical objects extending from the location icon in the graphical user interface.

[0045] In some embodiments, the graphical object representing the one or more search areas or the one or more geographic fences is a polygon. In some embodiments, the graphical object representing the one or more search areas or the one or more geographic fences is a triangle. In some embodiments, the graphical user interface includes a three-dimensional display and the graphical object representing the one or more search areas or the one or more geographic fences is a cone or other suitable shape.

[0046] Those skilled in the art will recognize that modifications to method 100 are within the scope of this disclosure. In some embodiments, method 100 includes at least one additional operation. In some embodiments, the order of operations in method 100 is adjusted.

[0047] 2 is a perspective view of a vehicle 200 according to some embodiments. The vehicle 200 is capable of implementing the method 100 (FIG. 1). In some embodiments, the vehicle 200 is capable of implementing the method 100 (FIG. 1) using a system 500 (FIG. 5) mounted on the vehicle. In some embodiments, the vehicle 200 is capable of implementing the method 100 (FIG. 1) based on receiving instructions from a system 500 (FIG. 5) that is remote or separable from the vehicle 200. In some embodiments where the system 500 (FIG. 5) is remote or separable from the vehicle 200, the vehicle 200 is configured to receive instructions to implement the method 100 (FIG. 1) wirelessly or via a wired connection.

[0048] Vehicle 200 includes one or more vehicle systems for implementing the operation of the vehicle. In some embodiments, the one or more vehicle systems include one or more of an infotainment system or a navigation system having one or more displays 201, one or more interior or exterior sensors 203, one or more interior or exterior cameras 205, and at least one camera 207 with a field of view outside the vehicle for taking pictures. In some embodiments, vehicle 200 includes one or more vehicle systems only in the front portion of the cabin. In some embodiments, vehicle 200 includes one or more vehicle systems in both the front portion of the cabin and the rear portion of the cabin.

[0049] FIG. 3 is a graphical user interface 300 according to one or more embodiments.

[0050] Graphical user interface 300 shows a geographic map 301 with a vehicle 303 on the right side of a road. In response to a query made by an occupant of vehicle 303 regarding a pointing and / or gaze direction toward the left side of the road, a search area represented by graphical object 305 expands to encompass potential objects of interest on the left side of the road, such as a coffee cafe 307, a clothing store 309, and a park 311.

[0051] When a query is made regarding pointing and / or gaze direction toward the right side of the road, the virtual search area represented by graphical object 313 is smaller than the search area represented by graphical object 305, which extends toward the potential object of interest on the right side of the road from vehicle 303 to the left side of the road. The number of lanes to the left of vehicle 303 is greater than the number of lanes to the right of vehicle 303.

[0052] Similarly, if vehicle 303 is driving on the left side of the road, the first search area represented by graphical object 305 will be smaller than the search area represented by graphical object 313 .

[0053] In some embodiments, if the vehicle is traveling on a highway or other multi-lane road, the search area and geographic fence are increased or decreased accordingly based on the number of lanes between the vehicle 303 and the side of the road corresponding to the vehicle occupant's query based on the pointing and / or gaze direction. In some embodiments, the search area and / or the geographic fence corresponding to the search area extend in various directions from the vehicle, such as left and right, and the size of the search area is adjusted based on the number of lanes between the vehicle 303 and the left side of the road and the right side of the road. In some embodiments, if the road has an odd number of lanes and the vehicle 303 is located in the center lane, the first search area and the second search area are set to be equal in size. In some embodiments, the sizes of the first search area and the second search area are initially equal, but if one side of the road has objects farther from the edge of the road than the other side, optionally one of the first search area or the second search area is increased until a candidate object of interest is identified.

[0054] FIG. 4 is a graphical user interface 400 according to one or more embodiments.

[0055] Graphical user interface 400 includes an image of a candidate object of interest 401, which in this example is a "coffee cafe," and descriptive information 403 associated with the candidate object of interest. In this example, descriptive information 403 includes the name and address of the candidate object of interest. In some embodiments, graphical user interface 400 is displayed based on user input received via graphical user interface 300 (FIG. 3). In some embodiments, graphical user interface 400 is displayed simultaneously with graphical user interface 300.

[0056] The candidate object of interest 401 in the graphical user interface 400 is within the graphical object 305 in the graphical user interface 300 and was identified by searching the map data according to a geographic fence set based on a search area corresponding to the graphical object 305.

[0057] 5 is a block diagram of a system 500 for adjusting a search scope to identify objects of interest to a vehicle occupant, according to one or more embodiments. The system 500 includes a hardware processor 502 and a non-transitory computer-readable storage medium 504 that is encoded with, i.e., stores, computer program code 506, i.e., a set of executable instructions. The computer-readable storage medium 504 is also encoded with instructions 507 for interfacing with a manufacturing machine to produce memory arrays. The processor 502 is electrically coupled to the computer-readable storage medium 504 via a bus 508. The processor 502 is also electrically coupled to an input / output (I / O) interface 510 by the bus 508. A network interface 512 is also electrically coupled to the processor 502 via the bus 508. The network interface 512 is connected to a network 514 such that the processor 502 and the computer-readable storage medium 504 can connect to external elements via the network 514. Processor 502 is configured to execute computer program code 506 encoded on computer-readable storage medium 504 to enable system 500 to perform some or all of the operations described in method 100 (FIG. 1) or implemented by vehicle 200 (FIG. 2).

[0058] In some embodiments, processor 502 is a central processing unit (CPU), a multiprocessor, a distributed processing system, an application specific integrated circuit (ASIC), and / or other suitable processing unit.

[0059] In some embodiments, computer-readable storage medium 504 is an electronic, magnetic, optical, electromagnetic, infrared, and / or semiconductor system (or apparatus or device). For example, computer-readable storage medium 504 includes semiconductor or solid-state memory, magnetic tape, removable computer diskette, random access memory (RAM), read-only memory (ROM), rigid magnetic disk, and / or optical disk. In some embodiments using an optical disk, computer-readable storage medium 504 includes a compact disk-read-only memory (CD-ROM), a compact disk-read / write (CD-R / W), and / or a digital video disk (DVD).

[0060] In some embodiments, storage medium 504 stores computer program code 504 configured to cause system 500 to perform some or all of the operations as described in method 100 ( FIG. 1 ) or implemented by vehicle 200 ( FIG. 2 ). In some embodiments, storage medium 504 also stores information used to perform some or all of the operations as described in method 100 ( FIG. 1 ) or implemented by vehicle 200 ( FIG. 2 ), as well as information generated during the performance of some or all of the operations as described in method 100 ( FIG. 1 ) or implemented by vehicle 200 ( FIG. 2 ), such as input data parameters 516, user profile parameters 518, notification data parameters 520, vehicle state parameters 522, and / or a set of executable instructions for performing some or all of the operations as described in method 100 ( FIG. 1 ) or implemented by vehicle 200 ( FIG. 2 ).

[0061] In some embodiments, storage medium 504 stores instructions 507 for interfacing with an external device, such as a mobile device, that enable processor 502 to generate or receive instructions readable by the external device during performance of some or all of the operations described in method 100 (FIG. 1) or implemented by vehicle 200 (FIG. 2).

[0062] System 500 includes an I / O interface 510. I / O interface 510 is coupled to external circuitry. In some embodiments, I / O interface 510 includes a keyboard, keypad, mouse, trackball, trackpad, touchscreen, and / or cursor direction keys for communicating information and commands to processor 502.

[0063] System 500 also includes a network interface 512 coupled to processor 502. Network interface 512 enables system 500 to communicate with a network 514 to which one or more other computer systems are connected. Network interface 512 may include a wireless network interface such as WiFi, Bluetooth, WiMAX, GPRS, or WCDMA; a wired network interface such as a LAN, Ethernet, WAN, USB, or IEEE-1394; or other suitable network interface. In some embodiments, some or all of the operations described in method 100 (FIG. 1) or implemented by vehicle 200 (FIG. 2) are implemented in more than one system 500, and information such as sensor data, window transmittance, forecast information, or vehicle status is exchanged between the various systems 500 via network 514.

[0064] Supplementary Note 1 An aspect of the present disclosure relates to a method. The method includes processing a query to identify an object outside the vehicle, the query being made by an occupant of the vehicle. The method also includes determining a focus direction of the occupant when the query is received. The method further includes detecting a number of lanes on a road on which the vehicle is traveling. In addition, the method includes, in response to the query, setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. The method also includes, in response to the query, setting one or more geographic fences on a geographic map corresponding to the one or more search areas for the vehicle. The method further includes searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. In addition, the method includes searching a database for information about candidate objects of interest within the one or more geographic fences. The method also includes, in response to the query, generating an output including information about the candidate objects of interest.

[0065] Supplementary Note 2 The method of Supplementary Note 1, wherein the one or more search areas include one of a first search area extending from the vehicle in a first direction or a second search area extending from the vehicle in a second direction different from the first direction, and the method further includes the steps of setting the one or more search areas to include one of the first search area or the second search area based on a focus direction of the occupant, and refraining from including the other of the first search area or the second search area based on a focus direction of the occupant.

[0066] Supplementary Note 3 3. The method of any one of claims 1 to 2, wherein the occupant's focus direction is based on a detected pointing gesture made by the occupant.

[0067] Supplementary Note 4 4. The method of any one of Supplementary Notes 1 to 3, wherein the occupant's focus direction is based on the occupant's detected line of sight.

[0068] Supplementary Note 5 The method of any one of Supplementary Notes 1 to 4, wherein the one or more search areas include at least two search areas, including a first search area extending in a first direction from the vehicle or a second search area extending in a second direction from the vehicle different from the first direction, and wherein the first search area and the second search area are all included within the search area corresponding to the one or more geographic fences for identifying the candidate object of interest.

[0069] Supplementary Note 6 The method of Supplementary Note 5, further comprising adjusting the size of the first search area and the size of the second search area based on one or more of a change in the occupant's focus direction or a detected lane change.

[0070] Supplementary Note 7 The method of any one of Supplementary Notes 1 to 6, wherein the one or more search areas include at least two search areas including a first search area extending from the vehicle toward a first side of the road and a second search area extending from the vehicle toward a second side of the road, the first search area extending beyond the first side of the road by a first distance and the second search area extending beyond the second side of the road by a second distance, the first distance being equal to the second distance.

[0071] Supplementary Note 8 The method of any one of Supplementary Notes 1 to 7, wherein the one or more search areas are each a triangle, each vertex of the triangle being located at the vehicle.

[0072] Supplementary Note 9 The method of any one of Supplementary Notes 1 to 8, wherein the one or more search areas are adapted to vehicle speed and increase in size from a first set size based on an increase in vehicle speed.

[0073] Supplementary Note 10 The method of any one of Supplementary Notes 1 to 9, wherein the one or more search areas are adapted to lane changes of the vehicle and increase or decrease in size from a first set size accordingly based on the detected lane changes.

[0074] Supplementary Note 11 An aspect of the present disclosure relates to an apparatus. The apparatus includes a processor and a memory, the memory having instructions stored therein that, when executed by the processor, cause the apparatus to process a query to identify an object outside the vehicle, the query being made by an occupant of the vehicle. The apparatus also performs the step of determining a focus direction of the occupant when the query is received. The apparatus further performs the step of detecting the number of lanes of a road on which the vehicle is traveling. Additionally, in response to the query, the apparatus performs the step of setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. In response to the query, the apparatus also performs the step of setting one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas. The apparatus further performs the step of searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. Additionally, the apparatus performs the step of searching a database for information about candidate objects of interest within the one or more geographic fences. In response to the query, the apparatus also performs the step of generating an output including information about the candidate objects of interest.

[0075] Supplementary Note 12 The apparatus described in Supplementary Note 11, wherein the one or more search areas include one of a first search area extending from the vehicle in a first direction or a second search area extending from the vehicle in a second direction different from the first direction, and the apparatus further performs the steps of setting the one or more search areas to include one of the first search area or the second search area based on the occupant's focus direction, and refraining from including the other of the first search area or the second search area based on the occupant's focus direction.

[0076] Supplementary Note 13 13. The apparatus of any one of claims 11 to 12, wherein the occupant focus direction is based on a detected pointing gesture made by an occupant of the vehicle.

[0077] Supplementary Note 14 14. The apparatus of any one of Supplementary Notes 11 to 13, wherein the occupant focus direction is based on a detected line of sight of the vehicle occupant.

[0078] Supplementary Note 15 The apparatus of any one of Supplementary Notes 11 to 14, wherein the one or more search areas include at least two search areas, including a first search area extending in a first direction from the vehicle or a second search area extending in a second direction from the vehicle different from the first direction, and wherein the first search area and the second search area are all included within the search area corresponding to the one or more geographic fences for identifying the candidate object of interest.

[0079] Supplementary Note 16 The apparatus of Supplementary Note 15, further comprising: adjusting the size of the first search area and the size of the second search area based on one or more of a change in the occupant's focus direction or a detected lane change.

[0080] Supplementary Note 17 17. The apparatus of any one of Supplementary Notes 11 to 16, wherein the one or more search areas include at least two search areas including a first search area extending from the vehicle toward a first side of the road and a second search area extending from the vehicle toward a second side of the road, the first search area extending beyond the first side of the road by a first distance, and the second search area extending beyond the second side of the road by a second distance, the first distance equal to the second distance.

[0081] Supplementary Note 18 The apparatus of any one of Supplementary Notes 11 to 17, wherein the one or more search areas are adapted to vehicle speed and increase in size from a first set size based on an increase in vehicle speed.

[0082] Supplementary Note 19 19. The apparatus of any one of Supplementary Notes 11 to 18, wherein the one or more search areas adapt to lane changes of the vehicle and increase or decrease in size from a first set size accordingly based on the detected lane changes.

[0083] Supplementary Note 20 An aspect of the present disclosure relates to a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor, cause an apparatus to process a query for identifying an object outside the vehicle, the query being made by an occupant of the vehicle. The apparatus also performs the step of determining a focus direction of the occupant when the query is received. The apparatus further performs the step of detecting the number of lanes of a road on which the vehicle is traveling. Additionally, in response to the query, the apparatus performs the step of setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and a side of the road, the side of the road being located forward in the focus direction of the occupant. In response to the query, the apparatus also performs the step of setting one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas. The apparatus further performs the step of searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences. Additionally, the apparatus performs the step of searching a database for information about candidate objects of interest within the one or more geographic fences. In response to the query, the apparatus also performs the step of generating an output including information about the candidate objects of interest.

Claims

1. 1. A processor-implemented method comprising: processing a query to identify an object external to the vehicle, the query being made by an occupant of the vehicle; determining a focus direction of an occupant when the inquiry is received; detecting the number of lanes on a road on which the vehicle is traveling; in response to the query, setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and the side of the road, the side of the road being located forward in a focus direction of the occupant; in response to the query, establishing one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas; searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences; searching a database for information about the candidate objects of interest within the one or more geographic fences; generating an output in response to said query, said output including information about said candidate objects of interest; A method comprising:

2. the one or more search areas include one of a first search area extending in a first direction from the vehicle or a second search area extending in a second direction from the vehicle that is different from the first direction; The method further comprises: setting the one or more search areas to include one of the first search area or the second search area based on a focus direction of the occupant; refraining from including the other of the first search area or the second search area based on a focus direction of the occupant; The method of claim 1 , comprising:

3. The method of claim 1 or 2, wherein the occupant focus direction is based on a detected pointing gesture made by an occupant of the vehicle.

4. The method of claim 1 or 2, wherein the occupant focus direction is based on a detected line of sight of the vehicle occupant.

5. 3. The method of claim 1 or 2, wherein the one or more search areas include at least two search areas, including a first search area extending from the vehicle in a first direction or a second search area extending from the vehicle in a second direction different from the first direction, and the first search area and the second search area are all included within the search area corresponding to the one or more geographic fences for identifying the candidate object of interest.

6. 6. The method of claim 5, further comprising adjusting the size of the first search area and the size of the second search area based on one or more of a change in the occupant's focus direction or a detected lane change.

7. 3. The method of claim 1 or 2, wherein the one or more search areas include at least two search areas including a first search area extending from the vehicle toward a first side of the road and a second search area extending from the vehicle toward a second side of the road, the first search area extending beyond the first side of the road by a first distance and the second search area extending beyond the second side of the road by a second distance, the first distance being equal to the second distance.

8. The method of claim 1 or 2, wherein each of the one or more search areas is a triangle, each vertex of the triangle being located at the vehicle.

9. 3. The method of claim 1 or 2, wherein the one or more search areas are adapted to vehicle speed, increasing in size from a first set size based on an increase in vehicle speed.

10. 3. The method of claim 1, wherein the one or more search areas are adapted to lane changes of the vehicle and are increased or decreased in size accordingly from a first set size based on detected lane changes.

11. 1. An apparatus comprising a processor and a memory, The memory has instructions stored therein that, when executed by the processor, cause the device to: processing a query to identify an object external to the vehicle, the query being made by an occupant of the vehicle; determining a focus direction of an occupant when the inquiry is received; detecting the number of lanes on a road on which the vehicle is traveling; in response to the query, setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and the side of the road, the side of the road being located forward in a focus direction of the occupant; in response to the query, establishing one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas; searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences; searching a database for information about the candidate objects of interest within the one or more geographic fences; generating an output in response to said query, said output including information about said candidate objects of interest; A device that performs the following.

12. the one or more search areas include one of a first search area extending in a first direction from the vehicle or a second search area extending in a second direction from the vehicle that is different from the first direction; The apparatus further comprises: setting the one or more search areas to include one of the first search area or the second search area based on a focus direction of the occupant; refraining from including the other of the first search area or the second search area based on a focus direction of the occupant; The apparatus of claim 11 , further comprising:

13. 13. The apparatus of claim 11 or 12, wherein the occupant focus direction is based on a detected pointing gesture made by an occupant of the vehicle.

14. 13. The apparatus of claim 11 or 12, wherein the occupant focus direction is based on a detected line of sight of the vehicle occupant.

15. 13. The apparatus of claim 11 or 12, wherein the one or more search areas include at least two search areas, including a first search area extending from the vehicle in a first direction or a second search area extending from the vehicle in a second direction different from the first direction, and wherein the first search area and the second search area are all included within the search area corresponding to the one or more geographic fences for identifying the candidate object of interest.

16. The apparatus further comprises:

16. The apparatus of claim 15, further comprising: adjusting a size of the first search area and a size of the second search area based on one or more of a change in the occupant's focus direction or a detected lane change.

17. 13. The apparatus of claim 11 or 12, wherein the one or more search areas include at least two search areas including a first search area extending from the vehicle toward a first side of the road and a second search area extending from the vehicle toward a second side of the road, the first search area extending beyond the first side of the road by a first distance and the second search area extending beyond the second side of the road by a second distance, the first distance being equal to the second distance.

18. 13. The apparatus of claim 11 or 12, wherein the one or more search areas are adapted to vehicle speed, increasing in size from a first set size based on an increase in vehicle speed.

19. 13. The apparatus of claim 11 or 12, wherein the one or more search areas are adapted to lane changes of the vehicle and are increased or decreased in size accordingly from a first set size based on detected lane changes.

20. processing a query to identify an object external to the vehicle, the query being made by an occupant of the vehicle; determining a focus direction of an occupant when the inquiry is received; detecting the number of lanes on a road on which the vehicle is traveling; in response to the query, setting one or more search areas extending from the vehicle based on the determined number of lanes between the vehicle and the side of the road, the side of the road being located forward in a focus direction of the occupant; in response to the query, establishing one or more geographic fences for the vehicle on a geographic map corresponding to the one or more search areas; searching map data associated with the geographic map to identify candidate objects of interest within the one or more geographic fences; searching a database for information about the candidate objects of interest within the one or more geographic fences; generating an output in response to said query, said output including information about said candidate objects of interest; A computer program that causes a processor to execute the following.

Images