Object determination method and vehicle

Abstract

The invention provides an object determination method. The object determination method comprises the following steps: determining a plurality of candidate objects outside a vehicle through an image acquisition device; obtaining geographic coordinates of the plurality of candidate objects; determining a relative position between the vehicle and each candidate object according to the geographic coordinates of the plurality of candidate objects; and according to the multiple relative positions, the target object in the target direction is determined from the multiple candidate objects, and the target direction is determined according to the target action of the passenger in the vehicle. The invention further provides a vehicle.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology and vehicle engineering technology, and in particular, to an object determination method and a vehicle. Background Art

[0002] With the rapid development of vehicle and computer technologies, the demand for personalized services for passengers has increased. For example, when a passenger is interested in an object outside the vehicle, the vehicle can provide information about the object to the user, which can enhance the passenger's interaction experience. However, affected by the external environment, it is difficult to accurately lock the object that the passenger is interested in, resulting in a reduced interaction experience for the passenger. Summary of the Invention

[0003] One aspect of the present disclosure provides an object determination method, including: determining a plurality of candidate objects outside the vehicle through an image acquisition device; obtaining the geographical coordinates of the plurality of candidate objects; determining the relative positions between the vehicle and each candidate object respectively according to the geographical coordinates of the plurality of candidate objects; and determining a target object in a target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action of a passenger inside the vehicle.

[0004] According to an embodiment of the present disclosure, obtaining the geographical coordinates of the candidate objects includes: obtaining the geographical coordinates of each of the plurality of unknown objects according to the geographical coordinates of known objects among the plurality of candidate objects and a target image, where the unknown objects are objects other than the known objects among the plurality of candidate objects, and the target image is an image corresponding to the plurality of candidate objects acquired by the image acquisition device.

[0005] According to an embodiment of the present disclosure, obtaining the geographical coordinates of each of the plurality of candidate objects according to the geographical coordinates of known objects among the plurality of candidate objects and the target image includes: determining the positional relationship between each unknown object and the known objects in the target image; determining a target map element corresponding to each unknown object from a target map according to the geographical coordinates of the known objects and the positional relationship; and determining the geographical coordinates of each unknown object according to the coordinate information corresponding to each target map element.

[0006] According to an embodiment of the present disclosure, the relative positions between the vehicle and each candidate object respectively include at least one of the following: two-dimensional relative position relationship, three-dimensional relative position relationship.

[0007] According to an embodiment of the present disclosure, determining the relative position between the vehicle and each candidate object respectively according to the geographical coordinates of a plurality of candidate objects includes: calculating the geographical distance between the candidate object and the vehicle according to the geographical coordinates of the candidate object and the geographical coordinates of the vehicle, where the geographical coordinates of the vehicle are obtained from a locator installed on the vehicle; calculating the target distance from the image acquisition device to the candidate object according to the preset distance between the locator and the image acquisition device and the geographical distance; and obtaining the three-dimensional relative position relationship between each candidate object and the vehicle according to the target distance and the internal reference information of the image acquisition device.

[0008] According to an embodiment of the present disclosure, determining the relative position between the vehicle and each candidate object respectively according to the geographical coordinates of a plurality of candidate objects includes: determining the first three-dimensional coordinates of the vehicle according to the geographical location of the vehicle and the terrain of the vehicle driving area; determining the second three-dimensional coordinates of each candidate object according to the three-dimensional map information and the plurality of candidate objects; and determining the three-dimensional relative position relationship between the vehicle and each candidate object respectively according to the first three-dimensional coordinates and the second three-dimensional coordinates.

[0009] According to an embodiment of the present disclosure, the above method further includes: determining a target direction in response to a target action performed by a passenger in the vehicle, where the target action includes at least one of the following: a window clicking action, a hand pointing action, and a gaze action.

[0010] According to an embodiment of the present disclosure, the above method further includes: obtaining the associated information of the target object; and outputting the associated information through an output device inside the vehicle.

[0011] Another aspect of the present disclosure provides a vehicle, including: a vehicle body; an image acquisition device for acquiring a target image outside the vehicle including a plurality of candidate objects; an action recognition device for acquiring the target actions of passengers in the vehicle; and a processing unit for obtaining the geographical coordinates of the plurality of candidate objects according to the target image; determining the relative position between the vehicle and each candidate object respectively according to the geographical coordinates of the plurality of candidate objects; and determining a target object in the target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action.

[0012] According to an embodiment of the present disclosure, the above vehicle further includes: a locator for acquiring the geographical coordinates of the vehicle body; and an output device for outputting the associated information of the target object.

[0013] Another aspect of the present disclosure provides an object recognition device, including: a first determination module configured to determine a plurality of candidate objects outside the vehicle through an image acquisition device; an acquisition module configured to acquire the geographical coordinates of the plurality of candidate objects; a second determination module configured to determine the relative positions between the vehicle and each of the candidate objects respectively according to the geographical coordinates of the plurality of candidate objects; and a third determination module configured to determine a target object in a target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action of a passenger inside the vehicle.

[0014] Another aspect of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the object determination method as above.

[0015] Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions, which are used to implement the object determination method as above when executed.

[0016] Another aspect of the present disclosure provides a computer program, where the computer program includes computer-executable instructions, which are used to implement the object determination method as above when executed.

[0017] According to an embodiment of the present disclosure, by determining a plurality of candidate objects outside the vehicle through an image acquisition device, a plurality of candidate objects that can be seen from the perspective of the passenger can be initially obtained. The geographical coordinates of the plurality of candidate objects are acquired. Thus, the relative positions between the vehicle and each of the candidate objects respectively can be determined according to the geographical coordinates of the plurality of candidate objects. Since the target direction is determined according to the target action of the passenger inside the vehicle, and then the target object in the target direction is determined from the plurality of candidate objects according to the plurality of relative positions, the behavior information of the passenger is combined, the target object that the passenger is interested in is accurately locked, and the interactive experience of the passenger is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To understand the present disclosure and its advantages more completely, reference will now be made to the following description in conjunction with the accompanying drawings, where:

[0019] Figure 1A Schematically shows an application scenario of the object determination method according to an embodiment of the present disclosure;

[0020] Figure 1B Schematically shows another application scenario of the object determination method according to an embodiment of the present disclosure;

[0021] Figure 1C Schematically shows yet another application scenario of the object determination method according to an embodiment of the present disclosure;

[0022] Figure 2 Schematically shows a flowchart of an object determination method according to an embodiment of the present disclosure;

[0023] Figure 3 Schematically shows a schematic diagram of determining the relative position between a vehicle and each candidate object respectively according to the geographical coordinates of a plurality of candidate objects according to an embodiment of the present disclosure;

[0024] Figure 4A Schematically shows a schematic diagram of a window click action according to an embodiment of the present disclosure;

[0025] Figure 4B Schematically shows a schematic diagram of a hand pointing action according to an embodiment of the present disclosure;

[0026] Figure 4C Schematically shows a schematic diagram of a gaze action according to an embodiment of the present disclosure;

[0027] Figure 4D Schematically shows a schematic diagram of an object determination method for a gaze action according to an embodiment of the present disclosure;

[0028] Figure 5 Schematically shows a block diagram of a vehicle according to an embodiment of the present disclosure;

[0029] Figure 6 Schematically shows a block diagram of an object recognition device according to an embodiment of the present disclosure;

[0030] Figure 7 Schematically shows a schematic block diagram of an example electronic device that can be used to implement the object determination method of the embodiments of the present disclosure. Detailed implementation manners

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0032] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0033] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0035] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, "a plurality" means two or more unless otherwise specifically defined.

[0036] Some block diagrams and / or flowcharts are shown in the drawings. It should be understood that some blocks or combinations of blocks in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create a device for implementing the functions / operations illustrated in these block diagrams and / or flowcharts.

[0037] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). Additionally, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by or in conjunction with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transport instructions. For example, a computer-readable medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of computer-readable media include: magnetic storage devices such as magnetic tapes or hard disk drives (HDDs); optical storage devices such as compact discs (CD-ROMs); memories such as random access memories (RAMs) or flash memories; and / or wired / wireless communication links.

[0038] Figure 1A An application scenario of the object determination method according to an embodiment of the present disclosure is schematically shown.

[0039] As Figure 1AAs shown, vehicle 110 is in motion. There are multiple objects outside the vehicle, such as the first building 120 and the second building 130. A passenger may want to know the associated information of the first building 120, such as its name, historical information, architectural style, etc.

[0040] Using a depth measurement sensor, the respective distances between the vehicle body 110 and the first building 120 and the second building 130 can be obtained.

[0041] However, when the vehicle 110 is driving on a bridge, the passenger can see the taller second building 130, but the lower first building 120 located under the bridge may not be in the visible area of the passenger's eyes. Due to the influence of the vehicle driving terrain and the height of the buildings, it is difficult to accurately lock the building that the passenger is interested in.

[0042] Figure 1B Another application scenario of the object determination method according to an embodiment of the present disclosure is schematically shown.

[0043] As Figure 1B shown, the first building 120 and the second building 130 are respectively located on the same side of the vehicle 110. However, from the passenger's perspective, there is an overlapping part between the first building 120 and the second building 130, that is, the first building 120 obscures a part of the architectural details of the second building 130. Due to the influence of the height between multiple buildings from the passenger's perspective, it is difficult to determine whether the passenger is interested in the first building 120 or the second building 130.

[0044] Figure 1C Another application scenario of the object determination method according to an embodiment of the present disclosure is schematically shown.

[0045] As Figure 1C shown, the respective distances between the first building 120 and the second building 130 and the vehicle body are the same. The vehicle 110 has driven past the first building 120, and the passenger may look back at the first building 120. Due to the influence of the passenger's behavior, it is also difficult to determine the building that the passenger is interested in.

[0046] Therefore, due to factors such as the vehicle driving terrain, building height, and passenger behavior, it is difficult to accurately lock the building that the passenger is interested in in the actual scenario.

[0047] In view of this, the present disclosure provides an object determination method, including: determining a plurality of candidate objects outside the vehicle through an image acquisition device; obtaining the geographical coordinates of the plurality of candidate objects; determining the relative positions between the vehicle and each candidate object according to the geographical coordinates of the plurality of candidate objects; and determining a target object in a target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action of the in-vehicle passenger.

[0048] According to an embodiment of the present disclosure, multiple candidate objects outside the vehicle are determined by an image acquisition device, and candidate objects that can be seen from the perspective of a passenger can be initially obtained. Geographical coordinates of the multiple candidate objects are acquired. Thus, the relative positions between the vehicle and each candidate object can be determined according to the geographical coordinates of the multiple candidate objects. Since the target direction is determined according to the target action of the in-vehicle passenger, and then according to the multiple relative positions, a target object in the target direction is determined from the multiple candidate objects, the behavior information of the passenger is combined to accurately lock the target object that the passenger is interested in, improving the interactive experience of the passenger.

[0049] Figure 2 A flowchart of an object determination method according to an embodiment of the present disclosure is schematically shown.

[0050] As Figure 2 shown, the object determination method includes operations S210 to S240.

[0051] In operation S210, multiple candidate objects outside the vehicle are determined by an image acquisition device.

[0052] Determining multiple candidate objects outside the vehicle by an image acquisition device may be receiving an image acquired by the image acquisition device and determining multiple candidate objects outside the vehicle according to the image acquired by the image acquisition device, or may be receiving information corresponding to multiple candidate objects outside the vehicle determined by the image acquisition device after processing the acquired image.

[0053] According to an embodiment of the present disclosure, a candidate object may be an object. For example, the object may be a building such as a store, a water tower, or an office building.

[0054] According to an embodiment of the present disclosure, the image acquisition device may be disposed on the outer surface of the vehicle.

[0055] Exemplarily, the image acquisition device may include multiple cameras disposed on the outer surface of the vehicle. The multiple cameras may be respectively disposed near the front of the vehicle, the chassis, the rear of the vehicle, etc., so as to capture multiple candidate objects outside during the driving of the vehicle.

[0056] For example, the image acquisition device may include a first camera disposed on the outer surface of the vehicle and a second camera disposed on the inner surface of the vehicle. The first camera may capture multiple candidate objects outside during the driving of the vehicle, and the second camera may acquire an action image of the passenger. However, it is not limited that the action image of the passenger is obtained by the image acquisition device.

[0057] For example, the external environment where the vehicle travels is an overpass (such as Figure 1AAs shown). It should be noted that due to the shooting angle of the image acquisition device, the first building 120 may not be included in multiple candidate objects. Therefore, by determining multiple candidate objects outside the vehicle through the image acquisition device, the interference of other objects is reduced, and the recognition efficiency of the target object is improved.

[0058] In operation S220, obtain the geographical coordinates of multiple candidate objects.

[0059] According to an embodiment of the present disclosure, the geographical coordinates may be longitude and latitude, or may be coordinate information in a map. According to the image captured by the image acquisition device, the geographical coordinates of multiple candidate objects can be obtained.

[0060] Exemplarily, use a machine learning algorithm to identify multiple candidate objects in the image captured by the image acquisition device, and obtain the respective name information of multiple candidate objects. According to the respective name information of multiple candidate objects, obtain the respective geographical coordinates of multiple candidate objects from a database, or query the respective geographical coordinates of multiple candidate objects from map information.

[0061] Exemplarily, according to the current position information of the vehicle, determine the position area to which multiple candidate objects belong. According to the relative position relationship between multiple candidate objects identified in the image captured by the image acquisition device, determine the respective geographical coordinates of multiple candidate objects from the position area.

[0062] In operation S230, according to the geographical coordinates of multiple candidate objects, determine the relative position between the vehicle and each candidate object respectively.

[0063] According to an embodiment of the present disclosure, according to the current position information of the vehicle and the geographical coordinates of multiple candidate objects, determine the relative position between the vehicle and each candidate object respectively.

[0064] For example, the current position information of the vehicle can be obtained from sensors on the vehicle.

[0065] According to an embodiment of the present disclosure, use a machine learning algorithm to identify the image captured by the image acquisition device to obtain the external environment in which the vehicle travels. The external environment may be a mountain road, an overpass, a tunnel, etc. According to the external environment in which the vehicle travels and the geographical coordinates of multiple candidate objects, the relative position between the vehicle and the candidate objects can be determined respectively.

[0066] For example, the external environment in which the vehicle travels is a tunnel, but the image acquisition device can capture multiple candidate objects, and it can be concluded that the vehicle is in the process of exiting or entering the tunnel. According to the geographical coordinates of multiple candidate objects and the information of the tunnel, the relative position between the vehicle and multiple candidate objects can be determined respectively.

[0067] For example, the external environment in which the vehicle travels is an overpass. Based on the object height of each of the multiple candidate objects and the geographical coordinates of each of the multiple candidate objects captured by the image acquisition device, the driving height of the vehicle and the geographical coordinates of the vehicle can be determined. Therefore, based on the geographical coordinates of the candidate objects, the object height, the geographical coordinates of the vehicle, and the driving height, the relative positions between the vehicle and the candidate objects are obtained.

[0068] According to an embodiment of the present disclosure, based on the geographical coordinates of multiple candidate objects and the pixel coordinates of the image captured by the image acquisition device, the relative positions between the vehicle and each of the multiple candidate objects can be determined.

[0069] For example, based on the geographical coordinates of multiple candidate objects and the pixel coordinates of the image captured by the image acquisition device, a conversion relationship between the camera coordinates and the world coordinates of the image acquisition device is established. Using the conversion relationship between the camera coordinates and the world coordinates and the spatial position of the image acquisition device on the vehicle, the position of the vehicle in the world coordinate system is calculated. Based on the position of the vehicle in the world coordinate system and the geographical coordinates of multiple candidate objects, the relative positions between the vehicle and the multiple candidate objects are obtained.

[0070] In operation S240, based on multiple relative positions, a target object in the target direction is determined from multiple candidate objects, and the target direction is determined according to the target action of the passengers in the vehicle.

[0071] According to an embodiment of the present disclosure, the target action of the passenger can be a head action, a hand action, a body rotation action, etc. The target direction can be the head orientation, the pointing direction of the hand, the orientation after the body rotation, etc. For example, the body rotation action can be a 10-degree rotation to the right, and the target direction can be the direction of the body orientation after a 10-degree rotation. In some embodiments, it can be to first obtain the target action of the passengers in the vehicle and then determine multiple candidate objects outside the vehicle through the image acquisition device, or it can be to first determine multiple candidate objects outside the vehicle through the image acquisition device and then obtain the target action of the passengers in the vehicle.

[0072] For example, as Figure 1BAs shown, the external environment where the vehicle is traveling is an overpass. Multiple candidate objects may include a first building 120 and a second building 130. First, using tracking and recognition technology, according to the object heights of the first building 120 and the second building 130 respectively, determine the unobstructed area where the second building is not blocked by the first building 120. According to the geographical coordinates of the first building 120 and the second building 130 respectively, determine the relative positions between the vehicle and the first building 120, and the relative positions between the vehicle and the unobstructed area of the second building 130. According to the relative positions between the vehicle and the first building 120, and the relative positions between the vehicle and the unobstructed area of the second building 130, determine the target object in the target direction from the first building area 120 and the second building 130.

[0073] For example, as Figure 1C shown, the target action of the passenger may be to turn back, then the first building 120 is in the target direction.

[0074] According to an embodiment of the present disclosure, determine the target direction according to the target action of the passenger in the vehicle to obtain the behavior information of the passenger. Since the relative positions between the vehicle and the candidate objects can comprehensively consider information such as the driving environment where the vehicle is located, the object height, and the behavior information of the passenger, therefore, according to multiple relative positions, determine the target object in the target direction from multiple candidate objects, which can accurately lock the object that the passenger is interested in.

[0075] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure, and application of the user information (the target action of the passenger in the vehicle) and other processes all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good customs. In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the authorization or consent of the user has been obtained.

[0076] According to an embodiment of the present disclosure, determine multiple candidate objects outside the vehicle through an image acquisition device, and initially obtain the candidate objects that can be seen from the passenger's perspective. Obtain the geographical coordinates of the multiple candidate objects. Thus, according to the geographical coordinates of the multiple candidate objects, the relative positions between the vehicle and each candidate object can be determined. Since the target direction is determined according to the target action of the passenger in the vehicle, and then according to multiple relative positions, determine the target object in the target direction from multiple candidate objects, which combines the behavior information of the passenger and accurately locks the target object that the passenger is interested in, improving the interactive experience of the passenger.

[0077] According to an embodiment of the present disclosure, obtaining the geographical coordinates of candidate objects includes: obtaining the geographical coordinates of each of multiple unknown objects based on the geographical coordinates of known objects among the multiple candidate objects and a target image, where the unknown objects are objects other than the known objects among the multiple candidate objects, and the target image is an image corresponding to the multiple candidate objects collected by an image acquisition device.

[0078] According to an embodiment of the present disclosure, a machine learning algorithm can be used to identify known objects from the target image collected by the image acquisition device; obtain the geographical coordinates of the known objects from a database or map information.

[0079] Exemplarily, determine the pixel coordinates of each of the multiple candidate objects from the target image, and determine the positional relationship between the multiple candidate objects. Based on the geographical coordinates of the known objects among the multiple candidate objects and the positional relationship between the multiple candidate objects, the geographical coordinates of each of the unknown objects can be determined respectively.

[0080] Exemplarily, determine the camera coordinates of the known objects from the target image, determine the conversion relationship from geographical coordinates to camera coordinates based on the geographical coordinates of the known objects and the camera coordinates of the known objects; determine the camera coordinates of the unknown objects from the target image; use the conversion relationship from geographical coordinates to camera coordinates to convert the camera coordinates of each of the multiple unknown objects into the geographical coordinates of each of the multiple unknown objects.

[0081] According to an embodiment of the present disclosure, obtaining the geographical coordinates of each of the multiple candidate objects based on the geographical coordinates of the known objects among the multiple candidate objects and the target image, so as to obtain the geographical coordinates of each of the multiple candidate objects in the target image, can avoid missing candidate objects, accurately determine the target object in the target direction, and thus accurately identify the target object of interest to the passenger.

[0082] According to an embodiment of the present disclosure, obtaining the geographical coordinates of each of the multiple candidate objects based on the geographical coordinates of the known objects among the multiple candidate objects and the target image includes: determining the positional relationship between each unknown object and the known objects in the target image; determining the target map element corresponding to each unknown object from the target map based on the geographical coordinates of the known objects and the positional relationship; determining the geographical coordinates of each of the unknown objects based on the coordinate information corresponding to each target map element.

[0083] Exemplarily, determine multiple unknown objects within the first target range of the known objects according to the target image, determine multiple to-be-matched map elements within the second target range of the map element corresponding to the known objects from the map information, match the multiple unknown objects with the to-be-matched map elements, and determine the geographical coordinates of each of the unknown objects according to the matching relationship, where each target element has a corresponding one in the map information. The target map element can be a building element in the map.

[0084] According to an embodiment of the present disclosure, the positional relationship between an unknown object and a known object in a target image can be determined based on the pixel points corresponding to the unknown object and the pixel points corresponding to the known object in the target image.

[0085] According to an embodiment of the present disclosure, the coordinate information corresponding to a target map element can be the camera coordinate or the geographical coordinate of the target element.

[0086] Exemplarily, the conversion relationship of converting camera coordinates into geographical coordinates can be utilized to convert the positional relationship between the unknown object and the known object in the target image, so as to obtain the positional relationship between the unknown object and the known object in the world coordinates. According to the geographical coordinates of the known object and the positional relationship between the unknown object and the known object in the world coordinates, multiple candidate geographical coordinates of the unknown object are determined. According to the target map element corresponding to the unknown object, the geographical coordinate of the unknown object is determined from the multiple candidate geographical coordinates.

[0087] Exemplarily, the conversion relationship of converting camera coordinates into geographical coordinates can be utilized to convert the camera coordinates corresponding to the target map element into the geographical coordinates corresponding to the target map element. According to the geographical coordinates corresponding to each target map element, the geographical coordinates of each unknown object are determined respectively.

[0088] According to an embodiment of the present disclosure, by determining the positional relationship between each unknown object and the known object in the target image respectively; according to the geographical coordinates and the positional relationship of the known object, the target map element corresponding to each unknown object is determined from the target map, the object information of the unknown object can be obtained. Therefore, according to the coordinate information corresponding to each target map element, the geographical coordinates of each unknown object can be accurately determined, and the recognition rate of the target object is improved.

[0089] According to an embodiment of the present disclosure, the relative position between the vehicle and each candidate object includes at least one of the following: two-dimensional relative position relationship, three-dimensional relative position relationship.

[0090] According to an embodiment of the present disclosure, the two-dimensional relative position relationship can be the relative position relationship in a two-dimensional coordinate system (longitude, latitude), the relative position relationship in the two-dimensional camera coordinate system (x, y) of the image acquisition device, etc. The two-dimensional coordinate system of the specific two-dimensional relative position relationship is not limited.

[0091] Exemplarily, according to the positioning information of the vehicle and the geographical coordinates of each candidate object, the relative position relationship in the two-dimensional coordinate system (longitude, latitude) between the vehicle and each candidate object is obtained.

[0092] Exemplarily, based on the positioning information of the vehicle and the geographical coordinates of the known object, the relative position relationship between the known object and the vehicle in the two-dimensional coordinate system (longitude, latitude) is determined. Using the coordinate conversion relationship between the two-dimensional coordinate system (longitude, latitude) and the two-dimensional camera coordinate system (x, y), the relative position relationship between the known object and the vehicle in the two-dimensional coordinate system (longitude, latitude) is converted to obtain the relative position relationship between the known object and the vehicle in the two-dimensional camera coordinate system (x, y), so as to obtain the relative position relationship between the vehicle and each candidate object in the two-dimensional camera coordinate system (x, y).

[0093] According to an embodiment of the present disclosure, the three-dimensional relative position relationship may be a relative position relationship in the world coordinate system (longitude, latitude, height), a relative position relationship in the three-dimensional camera coordinate system (X, Y, Z) of the image acquisition device, etc. The three-dimensional coordinate system of the specific three-dimensional relative position relationship is not limited.

[0094] Exemplarily, the world three-dimensional coordinates of each candidate object can be determined according to the height of each candidate object and the two-dimensional coordinates in the two-dimensional coordinate system (longitude, latitude). The world three-dimensional coordinates of the vehicle are determined according to the positioning information of the vehicle and the height at which the vehicle travels. According to the world three-dimensional coordinates of each candidate object and the world three-dimensional coordinates, the relative position relationship between the vehicle and each candidate object in the world coordinate system (longitude, latitude, height) is determined.

[0095] Exemplarily, the image acquisition device may be a depth camera. Therefore, the three-dimensional camera coordinates of the candidate object. The X-axis in the three-dimensional camera coordinate system represents the horizontal direction in the camera's field of view; the Y-axis represents the vertical direction in the camera's field of view; the Z-axis represents the depth direction of the camera, that is, the Z-axis value represents the distance from the candidate object to the camera. The relative position between the center point of the vehicle and the depth camera can be obtained from the vehicle's component structure information library, so as to determine the three-dimensional camera coordinates of the vehicle. According to the three-dimensional camera coordinates of each candidate object and the three-dimensional camera coordinates of the vehicle, the relative position relationship between the vehicle and each candidate object in the three-dimensional camera coordinate system (X, Y, Z) is determined.

[0096] According to an embodiment of the present disclosure, determining the relative position between the vehicle and each candidate object according to the geographical coordinates of multiple candidate objects includes: calculating the geographical distance between the candidate object and the vehicle according to the geographical coordinates of the candidate object and the geographical coordinates of the vehicle, and the geographical coordinates of the vehicle are obtained from the locator installed on the vehicle; calculating the target distance from the image acquisition device to the candidate object according to the preset distance between the locator and the image acquisition device and the geographical distance; obtaining the three-dimensional relative position relationship between each candidate object and the vehicle according to the target distance and the internal parameter information of the image acquisition device.

[0097] According to an embodiment of the present disclosure, the internal parameter information of the image acquisition device may include one of the following: focal length, principal point, pixel size, radial distortion, tangential distortion, etc.

[0098] Figure 3 Schematically shows a schematic diagram of determining the relative positions between the vehicle and each candidate object according to the geographical coordinates of multiple candidate objects according to an embodiment of the present disclosure.

[0099] As Figure 3 shown, the image acquisition device 111 and the locator 112 are respectively installed on the vehicle 110. The candidate object 120 outside the vehicle is determined by using the image acquisition device 111. The geographical coordinates of the vehicle are obtained by using the locator 112.

[0100] According to the geographical coordinates of the candidate object and the geographical coordinates of the vehicle, the geographical distance L1 between the candidate object 120 and the vehicle 110 is calculated. According to the preset distance L2 between the locator 112 and the image acquisition device 111 and the geographical distance L1, the target distance L3 from the image acquisition device 111 to the candidate object 120 is calculated.

[0101] Exemplarily, according to the geographical coordinates of the vehicle and the height of the locator, the world coordinates of the vehicle are determined. According to the preset distance and the world coordinates of the vehicle, the world coordinates of the image acquisition device are determined; according to the geographical coordinates of the candidate object and the height of the candidate object, the world coordinates of the candidate object are determined; based on the world coordinates of the image acquisition device and the world coordinates of the candidate object, the target distance is calculated.

[0102] Exemplarily, according to the target distance and the internal parameter information of the image acquisition device, the three-dimensional relative position relationship between each candidate object and the vehicle is calculated through the camera imaging principle and geometric relationship.

[0103] According to an embodiment of the present disclosure, through the preset distance between the locator and the image acquisition device, and the geographical distance between the candidate object and the vehicle, the target distance from the image acquisition device to the candidate object can be calculated. Then, by using the target distance and the internal parameter information of the image acquisition device, each candidate object and the vehicle can be accurately projected into the same space, and the three-dimensional relative position relationship between each candidate object and the vehicle can be obtained. Therefore, according to the three-dimensional relative position relationship between each candidate object and the vehicle, the target object in the target direction can be accurately identified.

[0104] According to an embodiment of the present disclosure, determining the relative positions between the vehicle and each candidate object respectively based on the geographical coordinates of a plurality of candidate objects includes: determining the first three-dimensional coordinates of the vehicle according to the geographical location of the vehicle and the terrain of the vehicle driving area; determining the second three-dimensional coordinates of each candidate object according to the three-dimensional map information and the plurality of candidate objects; determining the three-dimensional relative position relationships between the vehicle and each candidate object respectively according to the first three-dimensional coordinates and the second three-dimensional coordinates.

[0105] According to an embodiment of the present disclosure, the terrain of the vehicle driving area may be a plain, a mountain road, a tunnel, an overpass, etc.

[0106] For example, the geographical location of the vehicle is 100 degrees longitude and 45 degrees latitude; the terrain of the vehicle driving area is an overpass, and the first three-dimensional coordinates of the vehicle are determined according to the height range of the overpass and the geographical location of the vehicle.

[0107] For example, the vehicle driving area is a tunnel, and the first three-dimensional coordinates of the vehicle are determined according to the length of the tunnel, a plurality of candidate objects in the target image, and the geographical location of the vehicle.

[0108] According to an embodiment of the present disclosure, the three-dimensional map information may include information such as the shape and object height of the candidate object. The second three-dimensional coordinates of the candidate object may be 101 degrees longitude, 45 degrees latitude, and the height is 10 meters. The object height may be determined according to the altitude of the candidate object and the object height of the candidate object.

[0109] According to an embodiment of the present disclosure, the first three-dimensional coordinates of the vehicle are determined according to the geographical location of the vehicle and the terrain of the vehicle driving area; the second three-dimensional coordinates of each candidate object are determined according to the three-dimensional map information and the plurality of candidate objects. Therefore, it is not necessary to perform coordinate conversion on the pixel coordinates in the target image captured by the image acquisition device, and the vehicle and the candidate objects can also be projected into the same space, that is, the world coordinate system, where the first three-dimensional coordinates and the second three-dimensional coordinates are both coordinates in the world coordinate system. The three-dimensional relative position relationships between the vehicle and each candidate object are determined according to the first three-dimensional coordinates and the second three-dimensional coordinates, thereby reducing the computing resources for coordinate conversion, and the first three-dimensional coordinates of the vehicle and the second three-dimensional coordinates of the candidate objects can also be obtained quickly and accurately, and further improving the accuracy of the three-dimensional relative position relationships between the vehicle and each candidate object.

[0110] According to an embodiment of the present disclosure, the above method further includes: determining a target direction in response to a target action performed by a passenger in the vehicle, where the target action includes at least one of the following: a window-clicking action, a hand-pointing action, and a gaze-action.

[0111] According to an embodiment of the present disclosure, an action recognition device may be installed in a vehicle to recognize a target action performed by a passenger in the vehicle. For example, the action recognition device may include a camera installed in the vehicle, which can collect the actions of the passenger in the vehicle. The action recognition device may also include a touch layer on the window, which can obtain the position where the passenger clicks on the window.

[0112] In some embodiments, the relative position between the vehicle and each candidate object is a two-dimensional relative position relationship. Determining a target object in a target direction from multiple candidate objects according to multiple relative positions includes: mapping the target direction to a two-dimensional direction in a two-dimensional plane, and determining the target object in the two-dimensional direction from multiple candidate objects.

[0113] Figure 4A A schematic diagram showing the action of clicking on the window according to an embodiment of the present disclosure is schematically illustrated.

[0114] As Figure 4A shown, for the action of a passenger in the vehicle clicking on the window, the position where the passenger clicks on the window can be obtained from the touch layer of the window.

[0115] Exemplarily, the target direction can be determined according to the line connecting the position where the passenger clicks on the window and the eye position of the passenger. Or the target direction can be determined according to the position of the action of the passenger clicking on the window and the hand position.

[0116] Figure 4B A schematic diagram showing the hand-pointing action according to an embodiment of the present disclosure is schematically illustrated.

[0117] Exemplarily, the hand-pointing of the hand-pointing action is determined as the target direction. The hand-pointing can be recognized by a processor in the action recognition device using a machine learning algorithm.

[0118] Figure 4C A schematic diagram showing the eye-gaze action according to an embodiment of the present disclosure is schematically illustrated.

[0119] Exemplarily, the gaze direction of the eye-gaze action is determined as the target direction. The gaze direction can be determined according to the head orientation and eye position of the passenger.

[0120] According to an embodiment of the present disclosure, the positions of the hand and head of the passenger can be in the same two-dimensional coordinate system or the same three-dimensional coordinate system as the position of the vehicle. Thus, the target object in the target direction can be determined from multiple candidate objects according to the relative position between the vehicle and each candidate object.

[0121] Figure 4D A schematic diagram showing a method for determining an object for the eye-gaze action according to an embodiment of the present disclosure is schematically illustrated.

[0122] As Figure 4DAs shown, the locator is installed on the vehicle and is used to obtain the geographical coordinates 410 of the vehicle.

[0123] The image acquisition device may include a first camera. The first camera is used to capture multiple candidate objects outside the vehicle to obtain a target image and determine multiple candidate objects outside the vehicle. Obtain the geographical coordinates 420 of the multiple candidate objects.

[0124] The action recognition device may be a second camera. The second camera is installed inside the vehicle and is used to recognize the gaze action. The second camera may be a depth camera and can obtain the three-dimensional camera coordinates 430 of the passenger's eyes.

[0125] According to the geographical coordinates 410 of the vehicle and the geographical coordinates 420 of the multiple candidate objects, calculate the geographical distance between the candidate objects and the vehicle. According to the preset distance between the locator and the first camera and the geographical distance, calculate the target distance from the image acquisition device to the candidate objects through space; based on the camera imaging principle and geometric relationship, according to the target distance and the internal parameter information of the image acquisition device, obtain the three-dimensional relative position relationship 440 between each candidate object and the vehicle.

[0126] Unify the three-dimensional camera coordinates 430 of the passenger's eyes and the three-dimensional relative position relationship 440 between each candidate object and the vehicle, and the target object 450 in the target direction can be determined from the multiple candidate objects.

[0127] According to an embodiment of the present disclosure, in response to at least one of the actions of hitting the window, pointing with the hand, and gazing by the passenger in the vehicle, determine the target direction, the behavior of the passenger can be accurately obtained, so that the target object in the target direction can be accurately locked, improving the passenger experience.

[0128] According to an embodiment of the present disclosure, as Figure 1B shown, there are multiple candidate objects (such as the first building 120 and the second building 130) in the target direction, and accurate analysis can be combined with the body parts of the passenger, the height of the first building 120, and the height of the second building 130. For example, determine the unobstructed area where the second building 130 is not blocked by the first building 120. According to the gaze direction and / or hand pointing of the passenger's eyes, lock the area of interest of the passenger. Determine the building that coincides more with the area of interest from the unobstructed areas of the first building 120 and the second building 130 as the target object, so as to accurately lock the target object that the passenger is interested in.

[0129] According to an embodiment of the present disclosure, the above method further includes: obtaining the associated information of the target object; outputting the associated information through the output device inside the vehicle.

[0130] According to an embodiment of the present disclosure, the associated information may include the shape, appearance style, history and culture of the target object, etc. For example, the large model in the processor is used to identify the target image to obtain the name information of the target object, and according to the name information, the associated information of the target object is obtained from the database.

[0131] According to an embodiment of the present disclosure, the output device inside the vehicle may be a touch layer on the window, and the touch layer may be an electronic display screen. For example, the target action performed by the passenger in the vehicle may be a gaze action, and the projection point of the target object on the window is also in the target direction.

[0132] According to an embodiment of the present disclosure, by outputting the associated information of the target object, the associated information of the target object can be displayed to the passenger, enhancing the passenger experience.

[0133] Figure 5 A block diagram of a vehicle according to an embodiment of the present disclosure is schematically shown.

[0134] As Figure 5 shown, the vehicle 500 includes a vehicle body 510, an image acquisition device 520, an action recognition device 530, and a processing unit 540.

[0135] The image acquisition device 520 is configured to acquire a target image including a plurality of candidate objects outside the vehicle.

[0136] The action recognition device 530 is configured to acquire the target action of the passenger in the vehicle.

[0137] The processing unit 540 is configured to obtain the geographical coordinates of a plurality of candidate objects according to the target image; determine the relative positions between the vehicle and each candidate object respectively according to the geographical coordinates of the plurality of candidate objects; and determine the target object in the target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action.

[0138] According to an embodiment of the present disclosure, the above vehicle further includes: a locator for obtaining the geographical coordinates of the vehicle body; and an output device for outputting the associated information of the target object.

[0139] Figure 6 A block diagram of an object recognition device according to an embodiment of the present disclosure is schematically shown.

[0140] As Figure 6 shown, the object recognition device includes a first determination module 610, an acquisition module 620, a second determination module 630, and a third determination module 640.

[0141] The first determination module 610 is configured to determine a plurality of candidate objects outside the vehicle through the image acquisition device.

[0142] The acquisition module 620 is configured to acquire the geographical coordinates of a plurality of candidate objects.

[0143] The second determination module 630 is configured to determine the relative positions between the vehicle and each of the plurality of candidate objects according to the geographical coordinates of the plurality of candidate objects.

[0144] The third determination module 610 is configured to determine a target object in the target direction from the plurality of candidate objects according to the plurality of relative positions, where the target direction is determined according to the target action of the passengers in the vehicle.

[0145] It can be understood that the first determination module 610, the acquisition module 620, the second determination module 630, and the third determination module 640 can be combined and implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first determination module 610, the acquisition module 620, the second determination module 630, and the third determination module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application specific integrated circuit (ASIC), or can be implemented in any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in an appropriate combination of software, hardware, and firmware. Alternatively, at least one of the first determination module 610, the acquisition module 620, the second determination module 630, and the third determination module 640 can be at least partially implemented as a computer program module, and when the program is run by a computer, it can execute the functions of the corresponding module.

[0146] According to an embodiment of the present disclosure, the acquisition module 620 includes a first acquisition sub-module. The first acquisition sub-module is configured to obtain the geographical coordinates of each of the plurality of unknown objects according to the geographical coordinates of the known objects among the plurality of candidate objects and the target image, where the unknown objects are the objects other than the known objects among the plurality of candidate objects, and the target image is the image corresponding to the plurality of candidate objects acquired by the image acquisition device.

[0147] According to an embodiment of the present disclosure, the first acquisition sub-module includes a first determination unit, a second determination unit, and a third determination unit. The first determination unit is configured to determine the positional relationship between each unknown object and the known object in the target image; the second determination unit is configured to determine the target map element corresponding to each unknown object from the target map according to the geographical coordinates and the positional relationship of the known object; the third determination unit is configured to determine the geographical coordinates of each unknown object according to the coordinate information corresponding to each target map element.

[0148] According to an embodiment of the present disclosure, the relative position between the vehicle and each candidate object includes at least one of the following: two-dimensional relative position relationship, three-dimensional relative position relationship.

[0149] According to an embodiment of the present disclosure, the second determination module 630 includes a first calculation sub-module, a second calculation sub-module, and a second acquisition sub-module. The first calculation sub-module is configured to calculate the geographical distance between the candidate object and the vehicle according to the geographical coordinates of the candidate object and the geographical coordinates of the vehicle, and the geographical coordinates of the vehicle are obtained from a locator installed on the vehicle; the second calculation sub-module is configured to calculate the target distance from the image acquisition device to the candidate object according to the preset distance between the locator and the image acquisition device and the geographical distance; the second acquisition sub-module is configured to obtain the three-dimensional relative position relationship between each candidate object and the vehicle according to the target distance and the internal reference information of the image acquisition device.

[0150] According to an embodiment of the present disclosure, the second determination module 630 includes a first determination sub-module, a second determination sub-module, and a third determination sub-module. The first determination sub-module is configured to determine the first three-dimensional coordinates of the vehicle according to the geographical location of the vehicle and the terrain of the vehicle driving area; the second determination sub-module is configured to determine the second three-dimensional coordinates of each candidate object according to the three-dimensional map information and multiple candidate objects; the third determination sub-module is configured to determine the three-dimensional relative position relationship between the vehicle and each candidate object according to the first three-dimensional coordinates and the second three-dimensional coordinates.

[0151] According to an embodiment of the present disclosure, the above device further includes a fourth determination module. The fourth determination module is configured to determine a target direction in response to a target action performed by a passenger in the vehicle, where the target action includes at least one of the following: a window-clicking action, a hand-pointing action, a gaze action.

[0152] According to an embodiment of the present disclosure, the above device further includes an acquisition module and an output module. The acquisition module is configured to acquire the associated information of the target object; the output module is configured to output the associated information through an output device inside the vehicle.

[0153] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations, and necessary confidentiality measures are taken, and it does not violate public order and good customs. In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the authorization or consent of the user is obtained.

[0154] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0155] Figure 7A schematic block diagram of an example electronic device that can be used to implement the object determination method according to embodiments of the present disclosure is schematically shown.

[0156] The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0157] As Figure 7 shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 707 is also connected to the bus 704.

[0158] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0159] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as the virtual avatar driving method. For example, in some embodiments, the virtual avatar driving method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the virtual avatar driving method described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute the object determination method by any other suitable means (e.g., by means of firmware).

[0160] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program code is executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0164] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. Among them, the server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0166] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined or / and combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0167] Although the present disclosure has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art should understand that various changes in form and detail can be made to the present disclosure without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above embodiments, but should be determined not only by the appended claims but also by the equivalents of the appended claims.

Claims

1. A method for determining an object, comprising: determining a plurality of candidate objects outside the vehicle by an image acquisition device; Obtaining geographic coordinates of a plurality of candidate objects; Determining the relative position between the vehicle and each of the candidate objects according to the geographic coordinates of the plurality of candidate objects; According to the plurality of relative positions, the target object in a target direction is determined from the plurality of candidate objects, and the target direction is determined according to a target action of a passenger in the vehicle.

2. According to the method of claim 1, obtaining the geographic coordinates of the candidate object comprises: According to the geographic coordinates of known objects among the multiple candidate objects and the target image, the geographic coordinates of each of the multiple unknown objects are obtained, the unknown objects are objects other than the known objects among the multiple candidate objects, and the target image is the image corresponding to the multiple candidate objects acquired by the image acquisition device.

3. The method according to claim 2, wherein: The step of obtaining the geographic coordinates of each of the plurality of candidate objects according to the geographic coordinates of the known objects among the plurality of candidate objects and the target image comprises: Determine the positional relationship between each of the unknown objects and the known objects in the target image; Determining, from the target map, a target map element corresponding to each of the unknown objects according to the geographic coordinates of the known objects and the positional relationship; The geographic coordinates of each unknown object are determined according to the coordinate information corresponding to each target map element.

4. The method according to claim 1, wherein: The relative position between the vehicle and each of the candidate objects includes at least one of the following: Two-dimensional relative position relationship, three-dimensional relative position relationship.

5. The method according to claim 4, wherein: The determining the relative position between the vehicle and each of the candidate objects according to the geographic coordinates of the plurality of candidate objects comprises: Calculating the geographical distance between the candidate object and the vehicle according to the geographical coordinates of the candidate object and the geographical coordinates of the vehicle, wherein the geographical coordinates of the vehicle are obtained from a locator installed on the vehicle; Calculating a target distance from the image acquisition device to the candidate object according to a preset distance between the locator and the image acquisition device and the geographic distance; The three-dimensional relative position relationship between each of the candidate objects and the vehicle is obtained according to the target distance and the internal parameter information of the image acquisition device.

6. The method according to claim 4, wherein: The determining the relative position between the vehicle and each of the candidate objects according to the geographic coordinates of the plurality of candidate objects comprises: Determining a first three-dimensional coordinate of the vehicle according to the geographic location of the vehicle and the terrain of the vehicle driving area; Determine a second three-dimensional coordinate of each candidate object according to the three-dimensional map information and the plurality of candidate objects; The three-dimensional relative position relationship between the vehicle and each of the candidate objects is determined according to the first three-dimensional coordinates and the second three-dimensional coordinates.

7. The method according to claim 1, wherein: The method further comprises: The target direction is determined in response to a target action performed by a passenger in the vehicle, wherein the target action includes at least one of the following: Clicking on the car window, pointing with the hand, and looking at the eyes.

8. The method according to claim 1, wherein: The method further comprises: Obtaining the associated information of the target object; The related information is outputted via an output device inside the vehicle.

9. A vehicle comprising: Vehicle body; An image acquisition device, used for acquiring a target image containing a plurality of candidate objects outside the vehicle; A motion recognition device for collecting target motions of passengers in the vehicle; A processing unit, configured to obtain geographic coordinates of a plurality of candidate objects according to the target image; Determining the relative position between the vehicle and each of the candidate objects according to the geographic coordinates of the plurality of candidate objects; According to the plurality of relative positions, the target object in a target direction is determined from the plurality of candidate objects, and the target direction is determined according to the target action.

10. The vehicle according to claim 9, wherein: The vehicle further comprises: A locator, used to obtain the geographic coordinates of the vehicle body; The output device is used to output the associated information of the target object.

Images