Positioning method, device, equipment, medium, product and vehicle

Abstract

The invention relates to a positioning method and device, equipment, a medium, a product and a vehicle, and the method comprises the steps: obtaining the first position information of target electronic equipment, and enabling the target electronic equipment to be currently carried by a target object; acquiring a panoramic image of the vehicle; and determining target position information of the target object based on the panoramic image and the first position information. Therefore, the position of the target object is determined in combination with the panoramic image and the position information of the target electronic equipment, that is, the image information of the panoramic image is utilized to assist the vehicle in positioning, the positioning precision is improved, and meanwhile, the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a positioning method, device, equipment, medium, product and vehicle. Background Art

[0002] Existing positioning methods include developing UWB key products based on ultra-wide band (UWB) technology to achieve vehicle-centered near-field positioning, thereby realizing actions such as welcoming guests, contactless unlocking, and contactless locking.

[0003] However, due to the complexity and diversity of real-world situations, the UWB key products currently in use have various factors that cause the UWB signal to weaken, resulting in low positioning accuracy and affecting user experience. Summary of the Invention

[0004] The embodiments of the present application provide a positioning method, apparatus, computer equipment, storage medium, computer program product, and vehicle, which determine the location of a target object by combining panoramic images and position information of a target electronic device. That is, the image information of the panoramic image is used to assist the vehicle in positioning, thereby improving positioning accuracy and enhancing the user experience.

[0005] An embodiment of the present application provides a positioning method, which is applied to a vehicle, wherein the vehicle is equipped with a target electronic device, including:

[0006] Acquiring first location information of the target electronic device, where the target electronic device is currently carried by the target object;

[0007] Acquiring a panoramic image of the vehicle;

[0008] Based on the panoramic image and the first position information, target position information of the target object is determined.

[0009] Accordingly, an embodiment of the present application provides a positioning device, which is applied to a vehicle, wherein the vehicle is equipped with a target electronic device, including:

[0010] A first location information acquisition module is configured to acquire first location information of a target electronic device, where the target electronic device is currently carried by a target object;

[0011] A panoramic image acquisition module, configured to acquire a panoramic image of the vehicle;

[0012] A target position information determination module is configured to determine the target position information of the target object based on the panoramic image and the first position information.

[0013] In addition, an embodiment of the present application also provides a computer device, including one or more processors and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the positioning method provided in the embodiment of the present application.

[0014] In addition, an embodiment of the present application further provides a storage medium, which stores a computer program. When the computer program is run on a computer device, the computer program is used to enable the computer device to execute any one of the positioning methods provided in the embodiments of the present application.

[0015] In addition, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which implements any positioning method provided in the embodiment of the present application when executed by a processor.

[0016] In addition, an embodiment of the present application also provides a vehicle, including the above-mentioned computer device.

[0017] In an embodiment of the present application, first location information of a target electronic device is obtained, and the target electronic device is currently carried by a target object; a panoramic image of a vehicle is obtained; and the target location information of the target object is determined based on the panoramic image and the first location information. In this way, the location of the target object is determined by combining the panoramic image and the location information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a schematic diagram of an implementation environment scenario of the positioning method provided in the embodiments of the present application;

[0020] Figure 2 This is a flowchart of a positioning method provided by an embodiment of the present application;

[0021] Figure 3 This is a schematic diagram of a scenario in which a positioning method provided by an embodiment of the present application determines the distance information of a key relative to an anchor point;

[0022] Figure 4 This is a schematic diagram of a scenario for determining the location information of a target electronic device in an existing positioning method provided by an embodiment of the present application;

[0023] Figure 5 This is a schematic diagram of a scene within the range that can be covered by the panoramic imaging system in the positioning method provided in one embodiment of the present application;

[0024] Figure 6 This is a schematic diagram of a scenario showing the position coordinate range of the panoramic image coverage area in the positioning method provided by one embodiment of the present application;

[0025] Figure 7 This is a schematic diagram of identifying an object in a positioning method provided by an embodiment of the present application;

[0026] Figure 8 This is a schematic diagram of a scenario in which a positioning method provided by an embodiment of the present application calibrates an image coordinate system and a vehicle coordinate system of a panoramic image;

[0027] Figure 9 This is a flowchart of a positioning method provided by an embodiment of the present application;

[0028] Figure 10 This is a schematic structural diagram of a positioning device provided by an embodiment of the present application;

[0029] Figure 11 It is a structural diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0031] In addition, the term "a plurality of" in the embodiments of the present application refers to two or more than two. The terms "first" and "second" in the embodiments of the present application are used to distinguish descriptions and should not be understood to imply relative importance.

[0032] The embodiments of the present application provide a positioning method, apparatus, computer device, storage medium, computer program product, and vehicle. The positioning apparatus can be integrated into a computer device, which can be a server or a terminal.

[0033] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0034] The terminal may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited thereto. The terminal and the server may be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions thereon.

[0035] See also Figure 1 , taking the positioning device integrated into the computer equipment as an example, Figure 1 This is a schematic diagram of an implementation scenario of the positioning method provided in an embodiment of the present application, wherein the computer device may be a terminal device, which obtains first location information of a target electronic device currently carried by a target object; obtains a panoramic image of the vehicle; and determines the target location information of the target object based on the panoramic image and the first location information. The target electronic device refers to a device configured in the vehicle that can be used to locate the target object, such as a key product using ultra-wideband (UWB) wireless communication technology.

[0036] It should be noted that Figure 1 The schematic diagram of the implementation environment scenario of the positioning method shown is merely an example. The implementation environment scenario of the positioning method described in the embodiments of this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation of the technical solutions provided in the embodiments of this application. Persons skilled in the art will appreciate that with the evolution of data processing and the emergence of new business scenarios, the technical solutions provided in this application are equally applicable to similar technical problems.

[0037] The solutions provided in the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0038] This embodiment will be described from the perspective of a positioning device. The positioning device may be integrated into a computer device, which may be a terminal and / or a server. This application does not impose any limitation on this.

[0039] See also Figure 2 , Figure 21 is a flow chart of a positioning method provided in an embodiment of the present application, wherein the positioning method is applied to a vehicle equipped with a target electronic device. The positioning method may include the following steps S101 to S103:

[0040] S101: Acquire first location information of a target electronic device, where the target electronic device is currently carried by a target object.

[0041] The target electronic device is a device that can be located by the vehicle's positioning system. In other words, the target electronic device is a UWB-enabled device that needs to be located and has been authenticated by the vehicle owner. Authentication can be performed using the vehicle owner's phone or watch.

[0042] The first location information is used to indicate the location of the target electronic device.

[0043] The specific type of the target electronic device can be adjusted according to actual conditions and is not limited in the present embodiment. For example, the target electronic device is a key, a bracelet, etc.

[0044] It should be noted that the vehicle is equipped with a target electronic device and a positioning system. The positioning system can determine the location of the target electronic device. The target electronic device is currently being carried by the target subject, separated from the vehicle. By locating the target electronic device, the vehicle's positioning system can preliminarily determine the target subject's location.

[0045] The vehicle's positioning system can be based on various UWB methods. The target electronic device can be a UWB key or wristband. If the target object is the vehicle owner, and the vehicle owner gets out of the vehicle carrying the UWB key or wristband, the vehicle's UWB positioning system can locate the target electronic device, such as the UWB key or wristband, and determine the vehicle owner's location.

[0046] There are many ways for the UWB positioning system to locate the position of the target electronic device, which is not limited by the embodiments of the present application. For example, the phase difference (Phase-Difference-of-Arrival, PDOA) given by the UWB chip is converted to obtain the angle of the target electronic device relative to the anchor point, and the distance information of the target electronic device relative to the anchor point is obtained by a two-way ranging method, and the position of the target electronic device is determined by combining the angle of the target electronic device relative to the anchor point and the distance information of the target electronic device relative to the anchor point. For another example, the phase difference given by the UWB chip is converted to obtain the angle of the target electronic device relative to the anchor point, and the distance information of the target electronic device relative to the anchor point is obtained by a one-way ranging method, and the position of the target electronic device is determined by combining the angle of the target electronic device relative to the anchor point and the distance information of the target electronic device relative to the anchor point.

[0047] For example, assuming that the target electronic device is a key, the location of the key can be determined in the following manner.

[0048] The distance information of the key relative to the anchor point is determined by the following formula:

[0049]

[0050] Where d is the distance between the key and the anchor point, t1 is the time when the anchor point initiates the ranging request, t2 is the time when the key receives the request, t3 is the response time of the key reply, t4 is the time when the base station receives the response, and c is the speed of light.

[0051] See also Figure 3 , θ is the incident angle of electromagnetic wave, is the target quantity, and the antenna spacing d T is a fixed value and a known quantity. Antennas A and B are two antennas at the same anchor point, with the distance d between them being half a wavelength. For the CH9 frequency band, d is only 18 mm. The two antennas at this anchor point jointly receive the electromagnetic waves transmitted by the key.

[0052] The electromagnetic wave propagation distance difference p can be expressed according to the following formula:

[0053] p=d T ×sinθ

[0054] The flight time difference t can also be expressed as follows:

[0055]

[0056] Where c is the speed of light.

[0057] The phase difference Δθ can also be expressed as follows:

[0058] Δθ=ωt

[0059] Where ω is the angular velocity and the phase difference is a known quantity provided by the UWB chip.

[0060] Combining the above formula, the angle of the key relative to the anchor point can be determined as follows:

[0061]

[0062] Where f is the frequency of the electromagnetic wave.

[0063] S102: Acquire a panoramic image of the vehicle.

[0064] Among them, panoramic image refers to the image obtained by stitching and fusing multiple images of the vehicle's environment taken by a panoramic image camera.

[0065] S103: Determine target position information of the target object based on the panoramic image and the first position information.

[0066] The target location information is used to indicate the location information of the target object.

[0067] It should be noted that the target object is used to indicate the actual location of the target electronic device.

[0068] Specifically, see Figure 4 Existing UWB positioning systems primarily measure distance using UWB anchor points installed around and inside the vehicle. The distance measurements from multiple anchor points are combined to ultimately determine the location of a key, typically a UWB key product. In other words, existing technologies directly use the location of the target electronic device as the target object's location.

[0069] However, when the key is close to the vehicle and obstructed by sheet metal or a person, the UWB signal weakens, resulting in reduced positioning accuracy or even positioning failure, causing functional malfunction. For example, when the power door function requires recognition of the four door areas, when positioning accuracy decreases, the power door function will incorrectly determine whether the door can be opened and the location of obstacles (human bodies), resulting in a poor user experience. Therefore, relying solely on the location information obtained by the UWB system to determine the actual location information of the target electronic device can lead to low positioning accuracy in some cases.

[0070] The present embodiment uses the UWB system for positioning and combines the vehicle's panoramic image to determine the location of the target object. In other words, the panoramic image information is used to assist the UWB system in positioning, improving positioning accuracy and enhancing the user experience.

[0071] It should be noted that the UWB positioning system uses electromagnetic pulse signals to calculate flight time and detect phase difference for positioning. When the signal is blocked, the signal does not reach the receiving antenna directly, but arrives through reflection and other methods. Therefore, the flight time will become longer, resulting in a longer calculated distance. Similarly, the phase difference will become unstable due to factors such as reflection and refraction brought about by the complex spatial environment, resulting in fluctuations in the positioning results. The positioning information obtained by the panoramic imaging system is obtained directly through the image, and what is seen and obtained is more accurate than the information obtained by wireless transmission of electromagnetic signals.

[0072] As can be seen, the positioning method provided in the embodiments of the present application obtains first position information of a target electronic device, which is currently carried by a target object; obtains a panoramic image of the vehicle; and determines the target position information of the target object based on the panoramic image and the first position information. In this way, the position of the target object is determined by combining the panoramic image and the position information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0073] It should be noted that the panoramic image of the vehicle can be obtained in real time, and the panoramic image of the vehicle can also be obtained based on certain specific conditions. The specific conditions can be adjusted according to actual conditions, and the embodiments of this application do not limit this.

[0074] In some embodiments, a panoramic image of the vehicle is acquired in real time, and target position information of the target object is determined by combining the panoramic image and first position information of the target electronic device acquired in real time.

[0075] In some embodiments, if the target electronic device is located within the panoramic image coverage area of the vehicle, a panoramic image of the vehicle is acquired.

[0076] The panoramic image coverage area is used to indicate the range that can be covered by the vehicle's panoramic image system.

[0077] For example, see Figure 5 , Figure 5 : This is a schematic diagram of the range that can be covered by the panoramic imaging system provided in an embodiment of the present application. Among them, the range that can be covered by the panoramic imaging system includes the invalid area, the unlocking area, and the welcome area. The range that can be covered by the panoramic imaging system includes the panoramic image coverage area and the panoramic image uncovered area. The panoramic image coverage area includes the entire unlocking area and the part of the welcome area close to the unlocking area. The panoramic image uncovered area includes the invalid area and the part of the welcome area close to the invalid area. The invalid area indicates the range that cannot be covered by the panoramic imaging system. The welcome area indicates the area where the vehicle can be used to welcome guests, and the unlocking area indicates the area where the vehicle can be used to achieve contactless unlocking and contactless locking.

[0078] It should be noted that if the target electronic device is not within the vehicle's panoramic image coverage area, the UWB system's positioning result is directly output as the current target electronic device's location information. If the target electronic device is within the vehicle's panoramic image coverage area, the vehicle's panoramic image is acquired and combined with the panoramic image and the target electronic device's first location information to determine the current target electronic device's location information.

[0079] Since the UWB positioning system uses electromagnetic pulse signals to calculate flight time and detect phase difference for positioning, when the signal is blocked, the signal does not reach the receiving antenna directly, but arrives through reflection and other methods. Therefore, the flight time will be longer, resulting in a longer calculated distance. Similarly, the phase difference will become unstable due to factors such as reflection and refraction caused by the complex spatial environment, causing fluctuations in the positioning results. When the target electronic device is located in the panoramic image coverage area of the vehicle, a panoramic image of the vehicle is obtained. The panoramic image and the first position information of the target electronic device are combined to determine the current position information of the target electronic device. The positioning information obtained by the panoramic imaging system is obtained directly from the image. What is seen and what is obtained is more accurate than the information obtained by wirelessly transmitting electromagnetic signals.

[0080] Through the above steps, the panoramic image information is used to assist the UWB system in positioning, thereby improving positioning accuracy and enhancing the user experience.

[0081] In some embodiments, if the target electronic device is located within the panoramic image coverage area of the vehicle, before obtaining the panoramic image of the vehicle, it also includes: determining whether the target electronic device is located within the panoramic image coverage area of the vehicle based on the first position information and the position coordinate range of the panoramic image coverage area of the vehicle.

[0082] The panoramic image coverage area can be a polygonal area consisting of 4 or more points, which is defined by pre-calibrated data.

[0083] Specifically, if the coordinates corresponding to the first position information are within the position coordinate range of the vehicle's panoramic image coverage area, it is determined that the target electronic device is within the vehicle's panoramic image coverage area; otherwise, it is determined that the target electronic device is not within the vehicle's panoramic image coverage area.

[0084] For example, see Figure 6 , Figure 6 The following is a schematic diagram of the position coordinate range of the panoramic image coverage area provided by the embodiment of the present application. The position coordinate range of the panoramic image coverage area includes the geometric center of the vehicle body as the origin of the coordinate system, looking down at the vehicle, with the right side of the vehicle as the positive direction of the x-axis and the front of the vehicle as the positive direction of the y-axis. The x, y coordinates of the area covered by the panoramic image system are as follows: Figure 6 shown.

[0085] The location information provided by the UWB system is compared with the maximum and minimum values of the x, y coordinates of the panoramic imaging system's coverage area to determine whether the location is within the coverage range of the panoramic image, so as to facilitate the subsequent selection of an appropriate method to determine the target location information of the target object. The panoramic image information is used to assist the UWB system in positioning, improve positioning accuracy, and enhance the user experience.

[0086] In some embodiments, the above process of determining the target position information of the target object based on the panoramic image and the first position information may include: performing object position recognition based on the panoramic image to obtain a position recognition result; and determining the target position information of the target object based on the position recognition result and the first position information.

[0087] The position recognition result is used to indicate the result of object position recognition based on the panoramic image. Object position recognition is used to identify whether there is an object in the panoramic image and the location of the object. For example, see Figure 7 , the recognized object is a human-shaped object, and the position of the recognized object is the position of the human-shaped object in the panoramic image.

[0088] By performing object position recognition on the panoramic image, a position recognition result is obtained, and the position recognition result and the first position information are combined to determine the target position information of the target object. In this way, the panoramic image image information is used to assist the UWB system in positioning, improving positioning accuracy and enhancing the user experience.

[0089] In some embodiments, the position recognition result is second position information of at least one recognized object, and the second position information and the first position information are in the same coordinate system.

[0090] The second position information is used to indicate position information of the identification object and the target electronic device in the same coordinate system.

[0091] The aforementioned common coordinate system can be set based on actual circumstances. For example, the common coordinate system can be a vehicle coordinate system, i.e., a coordinate system established with the geometric center of the vehicle as the origin, with the vehicle's right side as the positive x-axis direction and the vehicle's front as the positive y-axis direction when viewed from above. For another example, the common coordinate system can be a pre-set coordinate system such that both the first location information of the target electronic device and the second location information of the identified object are displayed in the coordinate system.

[0092] By making the second position information of the identified object and the first position information of the target electronic device in the same coordinate system, the position information of the target object can be directly determined by the second position information and / or the first position information, without the need for other position conversion, thereby improving positioning efficiency.

[0093] In some embodiments, the above-mentioned process of performing object position recognition based on panoramic images and obtaining position recognition results includes: performing object recognition based on panoramic images to obtain at least one recognized object and third position information of the recognized object in the image coordinate system; determining the second position information of at least one recognized object based on pre-calibrated panoramic image parameters and the third position information.

[0094] Among them, the panoramic image person recognition algorithm refers to the image of the person captured by the panoramic camera, which usually only contains the person's feet, thighs and torso. This algorithm refers to a calculation method that identifies whether there is a human-shaped object in the picture based on these features.

[0095] The pre-calibrated panoramic image parameters refer to the correspondence between the image coordinate system of the panoramic image system and the vehicle coordinate system (or other identical coordinate systems).

[0096] Specifically, by calling the panoramic image character recognition algorithm, object recognition is performed based on the panoramic image to obtain at least one identified object and the third position information of the identified object in the image coordinate system; based on the pre-calibrated panoramic image parameters and the third position information, the second position information of at least one identified object is determined.

[0097] See also Figure 8 Assuming the vehicle coordinate system is the reference coordinate system, from a bird's-eye view perspective (a single camera perspective cannot calibrate the correspondence between pixels and positions), four calibration plates are laid on the ground. Using a measuring tool such as a tape measure, the true coordinates of the corner points of these four calibration plates (a total of 16 corner points) are determined. Then, based on the image provided by the panoramic camera, the correspondence between the specific 16 pixels in the image and these 16 corner points is determined. This allows the proportional relationship between the pixels and the true coordinate points to be determined, and the correspondence between each pixel in the image and the true coordinate point can be calculated. This correspondence is the calibration parameter. In theory, as long as there is a set of correspondences between the coordinates of image pixels and the real-world coordinate points, the real-world coordinate position can be calculated from the image. In practical applications, more points may be taken to reduce errors.

[0098] A corner point is a randomly selected point in the image coordinate system or vehicle coordinate system. If the 2D coordinates of the point are known, its corresponding 3D coordinates can be calculated. Similarly, if the 3D coordinates of the point are known, its corresponding 2D coordinates can be calculated.

[0099] For example, see Figure 8 Assuming that the image coordinate system coincides with the vehicle coordinate system (if they do not coincide, the two coordinate systems can be converted), the coordinates of a corner point a1 in the vehicle coordinate system are measured as (A 1x , A1y ), determine the pixel point (a 1x , a 1y ), calculate the magnification ratio k between the vehicle coordinate system and the image coordinate system according to the following formula x and k y :

[0100]

[0101]

[0102] Then the position of a known point in the image coordinate system is (h x , h y ), the real coordinates (H x , H y ):

[0103] H x =k x *h x

[0104] H y =k y *h y .

[0105] At this point, the position of a point in the vehicle coordinate system (ie, the second position information) can be determined through the position of the image pixel point (ie, the third position information).

[0106] Through the above steps, the third position information of the identified object in the panoramic image in the image coordinate system and the second position information in the vehicle coordinate system are quickly converted, so that the position information of the target object can be directly determined through the second position information and / or the first position information in the future without the need for other position conversions, thereby improving positioning efficiency.

[0107] In some embodiments, the above-mentioned determination of the target location information of the target object based on the location recognition result and the first location information includes: determining the candidate recognition object among the recognition objects that matches the location of the target electronic device based on the second location information and the first location information of each recognition object; and determining the target location information of the target object based on the second location information of the candidate recognition object.

[0108] The candidate recognition object indicates an object among the recognition objects that matches the position of the target electronic device.

[0109] Specifically, if a candidate identification object matches the position of the target electronic device among the other objects, it means that the panoramic imaging system has captured the target object carrying the target electronic device, and the target position information of the target object can be determined based on the second position information of the candidate identification object.

[0110] It should be noted that the UWB positioning system uses electromagnetic pulse signals to calculate flight time and detect phase difference for positioning. When the signal is blocked, the signal does not reach the receiving antenna directly, but arrives through reflection and other methods. Therefore, the flight time will become longer, resulting in a longer calculated distance. Similarly, the phase difference will become unstable due to factors such as reflection and refraction brought about by the complex spatial environment, resulting in fluctuations in the positioning results. The positioning information obtained by the panoramic imaging system is obtained directly through the image, and what is seen and obtained is more accurate than the information obtained by wireless transmission of electromagnetic signals.

[0111] When a candidate identification object is identified among the identification objects that matches the position of the target electronic device, the target position information of the target object is determined based on the second position information of the candidate identification object, so as to select a more accurate image for positioning, thereby improving positioning accuracy.

[0112] In some embodiments, if the position recognition result indicates that there is no recognition object, or there is no candidate recognition object among the recognition objects, the target position information of the target object is the first position information.

[0113] Through the above steps, the first position information of the target electronic device is directly used as the target position information of the target object, so as to deal with the situation where the identification object is not identified in the panoramic image coverage area, or even if the identification object is identified, after verification, the position of the identification object does not match the position of the target electronic device. The positioning result of the UWB system is directly used as the position of the target object to improve the positioning accuracy.

[0114] In some embodiments, the above process of determining candidate identification objects among the identification objects that match the position of the target electronic device based on the second position information and the first position information of each identification object may include: filtering out identification objects that do not match the position of the target electronic device based on the second position information and the first position information of each identification object to obtain candidate identification objects.

[0115] The candidate recognition object is used to indicate an object among the recognition objects that matches the position of the target electronic device.

[0116] Specifically, the first location information includes a plurality of first location point information, and the second location information includes a plurality of second location point information. Based on the plurality of second location point information and the plurality of first location point information of each identification object, identification objects that do not match the location of the target electronic device are filtered out to obtain candidate identification objects.

[0117] The first location information refers to the first location coordinate range of the target electronic device in the coordinate system, and the first location point information refers to the coordinate information of multiple coordinate points (i.e., first location points) within the first location coordinate range. In other words, the location coordinate range of the first location information includes multiple first location points and the coordinates of the first location points (i.e., first location point information).

[0118] The second position information refers to the second position coordinate range of the identified object in the corresponding coordinate system, and the second position point information refers to the coordinate information of multiple coordinate points (i.e., second position points) within the second position coordinate range. In other words, the position coordinate range of the second position information includes multiple second position points and the coordinates of the second position points (i.e., second position point information).

[0119] Through the second location information and the first location information, it can be determined whether the location of the identification object matches the location of the target electronic device. If not, the unmatched identification objects can be filtered and the candidate identification objects that match the location of the target electronic device are retained.

[0120] Through the above steps, unqualified identification objects are filtered out, and candidate identification objects that match the location of the target electronic device are retained. This allows the target object's target location information to be determined based on the location information of the candidate identification objects and the location information of the target electronic device. In this way, the panoramic image information is used to assist the UWB system in positioning, improving positioning accuracy and enhancing the user experience.

[0121] It should be noted that there are many filtering rules, which can be adjusted according to actual conditions and are not limited in the embodiments of this application.

[0122] In some embodiments, the above process of filtering out identification objects that do not match the position of the target electronic device based on the second position information and the first position information of each identification object to obtain candidate identification objects includes: calculating the distance between each identification object and the target electronic device based on the second position information and the first position information of each identification object, and / or determining the direction information of each identification object and the target electronic device relative to the vehicle; filtering out identification objects that do not match the position of the target electronic device based on the distance and / or direction information to obtain candidate identification objects.

[0123] The direction information is used to indicate the relative position of the identification object and the target electronic device to the vehicle. For example, if the location point provided by the panoramic imaging system (i.e., the identification object) is on the left side of the vehicle, and the location point provided by the UWB system (i.e., the target electronic device) is on the right side of the vehicle, then the direction of the identification object relative to the vehicle is different from the direction of the target electronic device relative to the vehicle, and the location point is excluded.

[0124] The distance is used to indicate the distance between the identification object and the target electronic device. For example, assuming the distance threshold is 0.5m, the distance between the location point provided by the panoramic imaging system and the location point provided by the UWB system is calculated in sequence. If the distance between the two points exceeds 0.5m, the location point is excluded.

[0125] Specifically, based solely on the distance between each identification object and the target electronic device, identification objects that do not match the location of the target electronic device can be filtered out to obtain candidate identification objects. Based solely on the direction information of each identification object and the target electronic device relative to the vehicle, identification objects that do not match the location of the target electronic device can be filtered out to obtain candidate identification objects. Furthermore, based on both the distance between each identification object and the target electronic device and the direction information of each identification object and the target electronic device relative to the vehicle, identification objects that do not match the location of the target electronic device can be filtered out to obtain candidate identification objects.

[0126] By identifying the distance and / or direction information between the object and the target electronic device, cross-verification is performed to assist the UWB system in positioning based on the verification results, thereby improving positioning accuracy and enhancing the user experience.

[0127] In some embodiments, if there are multiple candidate recognition objects.

[0128] Based on this, the above process of determining the target position information of the target object based on the second position information of the candidate identification object may include: determining the target candidate identification object according to the distance between each candidate identification object and the target electronic device; and determining the target position information of the target object as the second position information of the target candidate identification object.

[0129] The target candidate recognition object is used to indicate a candidate recognition object whose position information among multiple candidate recognition objects is more consistent with the position information of the target object.

[0130] There are many ways to determine the target candidate identification object, which can be adjusted according to actual conditions and are not limited in the embodiments of this application.

[0131] For example, the distance between the target candidate identification object and the target electronic device is the smallest.

[0132] By selecting the candidate identification object with the smallest distance from the target electronic device as the target candidate identification object, the target candidate identification object can be quickly determined, and the target position information of the target object can be quickly determined, thereby improving positioning efficiency.

[0133] For another example, an analysis may be performed based on the location information of multiple candidate recognition objects and the weights of the candidate recognition objects to obtain a measurement index of the candidate recognition objects, and the candidate recognition objects whose measurement index is close to the index threshold are used as target candidate recognition objects.

[0134] Through the above steps, a target candidate identification object is determined from a plurality of candidate identification objects, so that the target position information of the target object can be quickly determined based on the position information of the target candidate identification object, thereby improving positioning efficiency.

[0135] The following is an explanation based on a specific embodiment. Figure 9 , Figure 9 This is a flow chart of the positioning method provided in the embodiment of the present application, combined with Figure 9 It can be seen that the positioning method provided in the embodiment of the present application may include the following process: calculating the tag position through the UWB positioning module, wherein the tag position indicates the location of the target electronic device (ie, the key). Determine whether the tag position is within the area that can be photographed by the panoramic imaging system. If not, that is, the target (ie, the target object) is not within the panoramic image coverage area, output the positioning result of the UWB system, and then start the next round of positioning, and continue to use the UWB system for positioning. If so, that is, 2. the target is within the panoramic image coverage area, call the panoramic image character recognition algorithm to identify the position of the person. Use the panoramic imaging camera to continuously shoot the surrounding images of the vehicle, call the panoramic image character recognition algorithm to detect whether there are human-shaped objects around the vehicle, and record the number of objects identified. Depending on the number of objects, perform the following different operations:

[0136] (1) If only one object is identified, the position of the target object is calculated using pre-calibrated panoramic image parameters. Combined with the key location provided by the UWB system, cross-verification is performed to confirm whether the target object is the key location. If the two match, the position information provided by the panoramic image system is adopted as the final positioning result output. If the two do not match, the position information provided by the UWB system is adopted as the final positioning result output.

[0137] (2) Multiple objects are identified. The positions of all target objects are calculated using pre-calibrated panoramic image parameters. Combined with the key position given by the UWB system, cross-validation is performed to eliminate targets that are obviously inconsistent with the key position given by the UWB system. After the above selection process, if only one target object remains, the position information provided by the panoramic imaging system is output. If there are still multiple people in the nearby area, the straight-line distance between the coordinate points of each person and the position point provided by the UWB system is calculated and compared, and the object with the smallest distance is selected as the final positioning result output.

[0138] (3) Failure to identify an object indicates that the area captured by the panoramic camera is blocked by other objects or the light is too dark. For example, the driver carries a backpack, a doll, or other items that block his or her body features, resulting in recognition failure. The location information provided by the UWB system is directly output as the final result.

[0139] After completing the above steps, one round of positioning is completed. You can now start the next round of positioning and repeat the above steps.

[0140] Through the above embodiment, on the basis of positioning using the UWB system, when it is recognized that the car owner carries the located key into the area covered by the panoramic camera, by analyzing whether there are human objects in the picture (determining whether it is a person through the image of part of the body such as the feet or thighs), and the number of human objects, combined with the positioning information provided by the UWB system, other possible human objects are excluded, and the position of the human object carrying the located key device is accurately output.

[0141] In addition, by combining the image information provided by the panoramic camera, the UWB system is assisted in positioning, thereby providing more high-precision positioning capabilities and bringing a better user experience. To facilitate better implementation of the positioning method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above positioning method. The meanings of the terms are the same as those in the above positioning method. For specific implementation details, please refer to the description in the method embodiment.

[0142] For example, Figure 10 As shown, the positioning device is applied to a vehicle equipped with a target electronic device. The positioning device may include a first position information acquisition module 201, a panoramic image acquisition module 202, and a target position information determination module 203, as follows:

[0143] A first location information acquisition module 201 is configured to acquire first location information of a target electronic device, where the target electronic device is currently carried by a target object;

[0144] A panoramic image acquisition module 202 is used to acquire a panoramic image of the vehicle;

[0145] The target position information determining module 203 is configured to determine the target position information of the target object based on the panoramic image and the first position information.

[0146] As can be seen, the positioning device provided in the embodiment of the present application obtains the first position information of the target electronic device, which is currently carried by the target object, through the first position information acquisition module 201; the panoramic image acquisition module 202 obtains a panoramic image of the vehicle; and the target position information determination module 203 determines the target position information of the target object based on the panoramic image and the first position information. In this way, the panoramic image and the position information of the target electronic device are combined to determine the location of the target object. That is, the panoramic image image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0147] In one embodiment of the present application, the target location information determination module 203 includes:

[0148] A position recognition result determination submodule is used to perform object position recognition based on the panoramic image and obtain a position recognition result;

[0149] The target position information determination submodule is used to determine the target position information of the target object according to the position recognition result and the first position information.

[0150] In one embodiment of the present application, the position recognition result is second position information of at least one recognized object, and the second position information and the first position information are in the same coordinate system.

[0151] In one embodiment of the present application, the location recognition result determination submodule includes:

[0152] a third position information determining unit, configured to perform object recognition based on the panoramic image to obtain at least one recognized object and third position information of the recognized object in the image coordinate system;

[0153] The second position information determining unit is configured to determine second position information of at least one identified object according to pre-calibrated panoramic image parameters and third position information.

[0154] In one embodiment of the present application, the target location information determination submodule includes:

[0155] a candidate identification object determination unit, configured to determine, based on the second position information and the first position information of each identification object, a candidate identification object that matches the position of the target electronic device among the identification objects;

[0156] The target position information determining unit is configured to determine the target position information of the target object based on the second position information of the candidate recognition object.

[0157] In one embodiment of the present application, if the position recognition result indicates that there is no recognition object, or there is no candidate recognition object among the recognition objects, the target position information of the target object is the first position information.

[0158] In one embodiment of the present application, the candidate identification object determination unit includes:

[0159] The candidate identification object determination subunit is used to filter out identification objects that do not match the position of the target electronic device according to the second position information and the first position information of each identification object to obtain candidate identification objects.

[0160] In one embodiment of the present application, the candidate identification object determination subunit specifically includes:

[0161] Calculating the distance between each identified object and the target electronic device based on the second position information of each identified object and the first position information, and / or determining the direction information of each identified object and the target electronic device relative to the vehicle;

[0162] According to the distance and / or direction information, identification objects that do not match the position of the target electronic device are filtered out to obtain candidate identification objects.

[0163] In one embodiment of the present application, if there are multiple candidate identification objects.

[0164] Based on this, the target position information determining unit includes:

[0165] a target candidate identification object determination subunit, configured to determine a target candidate identification object based on the distance between each candidate identification object and the target electronic device;

[0166] The target position information determining subunit is configured to determine the target position information of the target object as the second position information of the target candidate identification object.

[0167] In one embodiment of the present application, the distance between the target candidate identification object and the target electronic device is the smallest.

[0168] In one embodiment of the present application, the panoramic image acquisition module 202 includes:

[0169] The panoramic image acquisition submodule is used to acquire a panoramic image of the vehicle if the target electronic device is located within the panoramic image coverage area of the vehicle.

[0170] In one embodiment of the present application, before the panoramic image acquisition submodule, the following is further included:

[0171] The judgment submodule is used to determine whether the target electronic device is located within the panoramic image coverage area of the vehicle based on the first position information and the position coordinate range of the panoramic image coverage area of the vehicle.

[0172] During specific implementation, the above modules can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. The specific implementation methods and corresponding beneficial effects of the above modules can be found in the previous method embodiments and will not be repeated here.

[0173] The present application also provides a computer device, such as Figure 11 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0174] The computer device may include one or more processing core processors 301, one or more storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will understand that Figure 11 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0175] Processor 301 is the control center of the computer device. It connects all components of the computer device using various interfaces and circuits. It executes computer programs and / or modules stored in memory 302 and accesses data stored in memory 302 to perform various computer functions and process data. Optionally, processor 301 may include one or more processing cores. Preferably, processor 301 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 301.

[0176] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and positioning by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as an image acquisition function, an image synthesis function, an automatic unlocking function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0177] The computer device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 303 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0178] The computer device may further include an input unit 304 , which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0179] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the computer device will load the executable files corresponding to one or more computer program processes into the memory 302 according to the following instructions, and the processor 301 will run the computer program stored in the memory 302 to implement various functions, such as:

[0180] Acquiring first location information of a target electronic device, where the target electronic device is currently carried by a target object;

[0181] Obtain panoramic images of the vehicle;

[0182] Based on the panoramic image and the first position information, target position information of the target object is determined.

[0183] As can be seen, the computer device provided in the embodiments of the present application obtains first location information of a target electronic device, which is currently carried by a target object; obtains a panoramic image of a vehicle; and determines the target location information of the target object based on the panoramic image and the first location information. In this way, the location of the target object is determined by combining the panoramic image and the location information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0184] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the detailed description of the positioning method above, which will not be repeated here.

[0185] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a storage medium and loaded and executed by a processor.

[0186] To this end, an embodiment of the present application provides a storage medium storing a computer program that can be loaded by a processor to execute the steps of any positioning method provided in the embodiment of the present application. For example, the computer program can execute the following steps:

[0187] Acquiring first location information of a target electronic device, where the target electronic device is currently carried by a target object;

[0188] Obtain panoramic images of the vehicle;

[0189] Based on the panoramic image and the first position information, target position information of the target object is determined.

[0190] As can be seen, the storage medium provided in the embodiments of the present application obtains first location information of a target electronic device, which is currently carried by a target object; obtains a panoramic image of a vehicle; and determines the target location information of the target object based on the panoramic image and the first location information. In this way, the location of the target object is determined by combining the panoramic image and the location information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0191] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the previous embodiments and will not be described in detail here.

[0192] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0193] Since the computer program stored in the storage medium can execute the steps of any positioning method provided in the embodiments of the present application, the beneficial effects that can be achieved by any positioning method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0194] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the above-mentioned positioning method.

[0195] As can be seen, the computer program product provided in the embodiments of the present application obtains first location information of a target electronic device, which is currently carried by a target object; obtains a panoramic image of a vehicle; and determines the target location information of the target object based on the panoramic image and the first location information. In this way, the location of the target object is determined by combining the panoramic image and the location information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0196] The present application also provides a vehicle comprising the aforementioned positioning device, the aforementioned computer device, or the aforementioned computer program product. The specific structure of the vehicle is not limited in this application. The specific implementations and corresponding beneficial effects of the aforementioned operations of the computer device are also applicable to the vehicle. For details, please refer to the detailed description of the positioning method above and will not be repeated here.

[0197] The vehicle is also equipped with a target electronic device, which can be carried by the target object to achieve separation from the vehicle. The vehicle's positioning system can determine the location of the target object by locating the target electronic device. For example, the target electronic device is a UWB key product, a UWB wristband product, etc. The target object is the vehicle owner. When the vehicle owner gets out of the vehicle carrying the UWB key product, UWB wristband product, etc., the vehicle's positioning system can determine the vehicle owner's location by locating the target electronic device such as the UWB key product, UWB wristband product, etc.

[0198] As can be seen, the vehicle provided in the embodiments of the present application obtains first position information of a target electronic device, which is currently carried by a target object; obtains a panoramic image of the vehicle; and determines the target position information of the target object based on the panoramic image and the first position information. In this way, the position of the target object is determined by combining the panoramic image and the position information of the target electronic device. That is, the panoramic image information is used to assist the vehicle in positioning, improve positioning accuracy, and enhance the user experience.

[0199] The above is a detailed introduction to a positioning method, device, computer equipment, storage medium, computer program product and vehicle provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A positioning method, characterized in that: Applied to a vehicle, the vehicle being equipped with a target electronic device, the method comprising: Acquiring first location information of the target electronic device, where the target electronic device is currently carried by the target object; Acquiring a panoramic image of the vehicle; Based on the panoramic image and the first position information, target position information of the target object is determined.

2. The positioning method according to claim 1, wherein: The determining the target position information of the target object based on the panoramic image and the first position information includes: Performing object position recognition based on the panoramic image to obtain a position recognition result; The target location information of the target object is determined according to the location recognition result and the first location information.

3. The positioning method according to claim 2, characterized in that: The position recognition result is second position information of at least one recognized object, and the second position information and the first position information are in the same coordinate system.

4. The positioning method according to claim 3, characterized in that: The performing object position recognition based on the panoramic image to obtain a position recognition result includes: performing object recognition based on the panoramic image to obtain at least one recognized object and third position information of the recognized object in the image coordinate system; The second position information of at least one of the identified objects is determined according to the pre-calibrated panoramic image parameters and the third position information.

5. The positioning method according to claim 3, characterized in that: The determining the target location information of the target object according to the location recognition result and the first location information includes: determining, based on the second location information of each of the identification objects and the first location information, a candidate identification object among the identification objects that matches the location of the target electronic device; The target position information of the target object is determined based on the second position information of the candidate recognition object.

6. The positioning method according to claim 5, characterized in that: If the position recognition result indicates that the recognition object does not exist, or if there is no candidate recognition object among the recognition objects, the target position information of the target object is the first position information.

7. The positioning method according to claim 5, characterized in that: The determining, based on the second location information of each of the identification objects and the first location information, a candidate identification object that matches the location of the target electronic device among the identification objects includes: According to the second position information of each identification object and the first position information, identification objects that do not match the position of the target electronic device are filtered out to obtain candidate identification objects.

8. The positioning method according to claim 7, characterized in that: The filtering out, based on the second location information of each identification object and the first location information, identification objects that do not match the location of the target electronic device to obtain candidate identification objects includes: Calculating the distance between each of the identified objects and the target electronic device based on the second position information of each of the identified objects and the first position information, and / or determining the direction information of each of the identified objects and the target electronic device relative to the vehicle; According to the distances and / or the direction information, identification objects that do not match the position of the target electronic device are filtered out to obtain candidate identification objects.

9. The positioning method according to claim 8, characterized in that: If there are multiple candidate recognition objects, determining the target location information of the target object based on the second location information of the candidate recognition objects includes: Determining a target candidate identification object based on the distance between each candidate identification object and the target electronic device; The target position information of the target object is determined as the second position information of the target candidate identification object.

10. The positioning method according to claim 8, characterized in that: The distance between the target candidate identification object and the target electronic device is the smallest.

11. The positioning method according to any one of claims 1 to 10, characterized in that: The obtaining of a panoramic image of the vehicle includes: If the target electronic device is located within the panoramic image coverage area of the vehicle, a panoramic image of the vehicle is acquired.

12. The positioning method according to claim 11, characterized in that: If the target electronic device is located within the panoramic image coverage area of the vehicle, before acquiring the panoramic image of the vehicle, the method further includes: It is determined whether the target electronic device is located within the panoramic image coverage area of the vehicle according to the first position information and the position coordinate range of the panoramic image coverage area of the vehicle.

13. A positioning device, characterized in that: Applied to a vehicle, the vehicle being equipped with a target electronic device, the apparatus comprising: A first location information acquisition module is configured to acquire first location information of a target electronic device, where the target electronic device is currently carried by a target object; A panoramic image acquisition module, configured to acquire a panoramic image of the vehicle; A target position information determination module is configured to determine the target position information of the target object based on the panoramic image and the first position information.

14. A computer device, characterized in that: The system comprises one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the positioning method according to any one of claims 1 to 12.

15. A storage medium, characterized in that: The method comprises a computer program, which is used to cause a computer device to execute the steps of the positioning method according to any one of claims 1 to 12 when the computer program is run on the computer device.

16. A computer program product, characterized in that The method comprises a computer program or instructions, which implements the steps of the positioning method according to any one of claims 1 to 12 when the computer program or instructions are executed by a processor.

17. A vehicle, characterized in that: The vehicle includes the computer device of claim 14.

Images