Object positioning method, electronic device and computer readable storage medium

By obtaining the environmental image and camera parameters, determining the alternative position and virtual camera parameters, and using three-dimensional model to render the image comparison, the problem of inaccurate positioning in harsh environments is solved, and fast and accurate object positioning is achieved, which is suitable for poor signal condition.

CN115307637BActive Publication Date: 2024-11-19SHANGHAI LICHI SEMICON LTD

Patent Information

Application Number
CN202210863847.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-21
Publication Date
2024-11-19
Estimated Expiration
2042-07-21

AI Technical Summary

Technical Problem

In harsh environments, cars or robots cannot accurately obtain environmental information with sensors, resulting in inaccurate positioning and reducing positioning reliability.

Method used

By obtaining the environmental image and camera parameters of the target object, select position information and virtual camera parameters are determined, and rendered based on the preset three-dimensional model rendering is generated, and compared with the real environment image to determine the position information of the target object.

Benefits of technology

It realizes the rapid and accurate positioning of the target object in harsh environments, improves the flexibility and scope of application of positioning, and is suitable for conditions of poor signal.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115307637B_ABST
    Figure CN115307637B_ABST
Patent Text Reader

Abstract

The present application discloses an object positioning method, an electronic device and a computer-readable storage medium, the method comprising: obtaining an environmental image collected by a target object through its own camera and the camera parameters for collecting the environmental image; determining the target object's alternative position information and the parameters of a virtual camera, the parameters of the virtual camera and the camera parameters of the environmental image having a first corresponding relationship; rendering and generating a rendering image corresponding to the alternative position information based on a preset three-dimensional model, the alternative position information and the parameters of the virtual camera; the three-dimensional model and the real space having a second corresponding relationship; determining the position information of the target object based on the environmental image, the rendered image and the alternative position information. In this way, the positioning of the target object can be realized simply and quickly, and the requirements for the environment in which the target object is located are lower, the flexibility is higher, and the scope of application is wider, for example, it can be applied to harsh conditions with poor signals such as tunnels and bridge holes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of object positioning, and in particular to an object positioning method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of intelligent driving technology, cars or robots have more and more intelligent functions, and the realization of many intelligent functions of cars or robots depends on the real-time and accurate positioning of the cars or robots. That is, in order to realize many intelligent functions of cars or robots, it is usually necessary to accurately determine the position of the car or robot on the road in real time.

[0003] At present, it is usually possible to obtain environmental information through various sensors carried by cars or robots, and then use the prior data collected in advance (such as visual high-precision feature maps, semantic high-precision feature maps, etc.) to calculate real-time positioning information through matching or graph optimization methods. For example, cars can use landmark-based car positioning technology to obtain environmental information through visual sensors such as cameras and lidar sensors, and send the environmental information to the vehicle controller of the car. The vehicle controller of the car can match the environmental information with the feature information in the database to determine the specific location and environment of the car.

[0004] However, because the sensors carried by the car or robot may not be able to obtain environmental information under harsh conditions, or the obtained environmental information is inaccurate, it is impossible to accurately position the car or robot, reducing the reliability of the car or robot positioning. Summary of the invention

[0005] In view of this, the embodiments of the present application provide an object positioning method, an electronic device, and a computer-readable storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0006] According to the first aspect of the present application, an embodiment of the present application provides an object positioning method, including: obtaining an environmental image captured by the target object through its own camera and the camera parameters for capturing the environmental image; determining alternative position information of the target object and the parameters of the virtual camera, the parameters of the virtual camera and the camera parameters of the environmental image having a first corresponding relationship; based on a preset three-dimensional model, the alternative position information and the parameters of the virtual camera, rendering and generating a rendered image corresponding to the alternative position information; the three-dimensional model and the real space have a second corresponding relationship; based on the environmental image, the rendered image and the alternative position information, determining the position information of the target object.

[0007] Optionally, determining candidate position information of the target object includes: acquiring position information of the target object at a first moment, where the first moment is a moment before a current moment; and determining the candidate position information of the target object based on the position information at the first moment.

[0008] Optionally, determining alternative position information of the target object based on the position information at the first moment includes: acquiring first moving speed information of the target object; calculating the moving distance of the target object based on the first moving speed information and time difference information between the current moment and the first moment; and predicting the position information of the target object at the current moment based on the position information at the first moment and the moving distance of the target object as the alternative position information of the target object.

[0009] Optionally, based on the environment image, the rendered image and the alternative position information, the position information of the target object is determined, including: comparing the environment image and the rendered image; when it is determined that the environment image and the rendered image meet a preset condition, determining the alternative position information as the position information of the target object; when it is determined that the environment image and the rendered image do not meet the preset condition, calculating the position offset information based on the environment image and the rendered image, updating the alternative position information based on the position offset information, and returning to the step of obtaining the environment image captured by the target object through its own camera and the camera parameters for capturing the environment image, until it is determined that the environment image and the rendered image meet the preset condition, and then determining the updated alternative position information as the position information of the target object.

[0010] Optionally, the position offset information includes time information when the target object captures the next frame of the environment image;

[0011] Correspondingly, after the candidate position information is updated based on the position offset information, the target object re-collects the environment image through its own camera in response to the time information.

[0012] Optionally, before comparing the environment image with the rendered image, the object positioning method further includes: determining model parameters corresponding to the candidate position information in the three-dimensional model; and processing the environment image based on the model parameters.

[0013] Optionally, after determining the position information of the target object based on the environment image, the rendered image and the alternative position information, the object positioning method also includes: obtaining second moving speed information of the target object; predicting the position information of the target object at a second moment based on the position information of the target object and the second moving speed information, as the alternative position information at the second moment, the second moment being a moment after the current moment; rendering and generating a rendered image corresponding to the alternative position information at the second moment based on a preset three-dimensional model, the alternative position information at the second moment and the parameters of the virtual camera, so as to generate a rendered image corresponding to the alternative position information at the second moment in advance.

[0014] Optionally, after determining the position information of the target object based on the environment image, the rendered image and the alternative position information, the object positioning method further includes: determining difference information of the three-dimensional model relative to the real space based on the environment image and the rendered image; and updating the three-dimensional model based on the difference information.

[0015] Optionally, the target object includes a plurality of cameras arranged at different positions.

[0016] Correspondingly, after determining the alternative position information of the target object, the object positioning method also includes: determining model parameters corresponding to the alternative position information in the three-dimensional model; based on the model parameters, selecting the environmental image captured by the camera at the target orientation and the camera parameters for capturing the environmental image for determining the position information of the target object.

[0017] According to the second aspect of the present application, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the object positioning method as in the first aspect or any embodiment of the first aspect.

[0018] According to the third aspect of the present application, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the object positioning method in the first aspect or any embodiment of the first aspect.

[0019] The object positioning method, electronic device and computer-readable storage medium provided in the embodiments of the present application obtain an environmental image captured by the target object through its own camera and the camera parameters of the environmental image; determine the target object's alternative position information and the parameters of the virtual camera, and the parameters of the virtual camera have a first corresponding relationship with the camera parameters of the environmental image; based on the preset three-dimensional model, the alternative position information and the parameters of the virtual camera, render and generate a rendered image corresponding to the alternative position information; the three-dimensional model and the real space have a second corresponding relationship; based on the environmental image, the rendered image and the alternative position information, determine the position information of the target object; in this way, in the virtual three-dimensional model matching the real space, a rendered image corresponding to the alternative position information is generated by rendering, and then the rendered image is matched with the environmental image actually taken by the target object to obtain the current accurate position information of the target object, which can simply and quickly realize the positioning of the target object, and has lower requirements on the environment in which the target object is located. Compared with obtaining environmental information through various sensors carried on the target object, and then using the prior data collected in advance to calculate the real-time positioning information through matching or graph optimization and other methods, it has higher flexibility and a wider range of application. For example, it can be applied to harsh conditions with poor signals such as tunnels and bridge holes.

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a flow chart of an object positioning method in an embodiment of the present application;

[0022] Figure 2 This is a schematic diagram of an application scenario in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of generating a rendered image in an embodiment of the present application;

[0024] Figure 4 This is a flow chart of step S104 in the embodiment of the present application;

[0025] Figure 5 This is a schematic diagram of a process for processing an environment image based on model parameters in an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0028] The present application embodiment provides an object positioning method, which can be applied to scenarios such as vehicle positioning on the road and robot indoor positioning. Figure 1 As shown, the object positioning method includes:

[0029] S101, obtaining an environment image captured by a target object through its own camera and camera parameters for capturing the environment image.

[0030] In this embodiment, the target objects include but are not limited to cars, robots, and intelligent unmanned vehicles.

[0031] In this embodiment, cameras can be set at multiple positions of the target object, and the images of the surrounding environment can be collected in real time. One, several or all images collected by the cameras can be selected as the environmental images collected by the target object for the location calculation of the target object. When collecting the environmental images, the camera parameters when collecting the environmental images should also be recorded in real time. The camera parameters include but are not limited to the shooting angle, the size of the field of view, etc.

[0032] S102, determining candidate position information of the target object and parameters of the virtual camera, wherein the parameters of the virtual camera and the camera parameters of the environment image have a first corresponding relationship.

[0033] In this embodiment, the candidate position information of the target object includes one or more candidate positions. Each candidate position corresponds to a parameter of a virtual camera. The virtual camera is a hypothetical camera in a virtual space corresponding to the real space where the target object is located. The parameters of the virtual camera and the camera parameters of the environment image have a first corresponding relationship, but the parameter sizes may be the same or different. The parameters of the virtual camera include but are not limited to shooting angle, field of view size, etc.

[0034] In some embodiments, determining the candidate position information of the target object includes: obtaining the position information of the target object at a first moment, where the first moment is a moment before the current moment; and determining the candidate position information of the target object based on the position information at the first moment. That is, the candidate position information of the target object can be predicted based on the position information of the target object before the current moment.

[0035] In one implementation, determining alternative position information of a target object based on position information at a first moment includes: acquiring first moving speed information of the target object; calculating a moving distance of the target object based on the first moving speed information and time difference information between a current moment and a first moment; and predicting the position information of the target object at a current moment as the alternative position information of the target object based on the position information at the first moment and the moving distance of the target object.

[0036] For example, Figure 2 As shown in FIG, before a target object, such as a car, enters a bridge tunnel, that is, at the first moment, its position information can be calculated by the global navigation satellite system carried by the car, such as Figure 2 Position 1 in the tunnel. When the car enters the tunnel, the global navigation satellite signal disappears, and the current position information of the car cannot be calculated by the global navigation satellite system, but the alternative position information of the car can be predicted based on the position information at the first moment. First, the moving distance L of the car from the first moment to the current moment can be calculated. The moving distance L can be calculated by the first moving speed information of the car and the time difference information between the current moment and the first moment. The predicted position of the car at the current moment is near position 2, and position 2 is the position of position 1 at the first moment after moving forward along the road by the moving distance L. According to the positioning accuracy, one or several target positions can be selected near position 2 as the alternative position information of the car, such as Figure 2 Alternative position 1, alternative position 2, and alternative position 3 are shown.

[0037] In this embodiment, by acquiring the position information of the target object at the first moment and predicting the candidate position information of the target object based on the position information at the first moment, a smaller number of candidate positions that are more closely matched with the position of the target object at the current moment can be obtained.

[0038] In some embodiments, the candidate positions of the target object can be directly selected in a virtual space corresponding to the real space, such as a three-dimensional model corresponding to the real space, so as to obtain the candidate position information. For example, when a car is driving on a road, several target positions can be selected as candidate positions on a virtual road corresponding to the road to obtain the candidate position information. For another example, when a robot moves indoors, several target positions can be selected as candidate positions in a virtual space corresponding to the indoor space to obtain the candidate position information.

[0039] In some embodiments, the candidate position information of the target object can be determined based on the environment image. For example, a car randomly takes an environment image while driving on the road, and the virtual camera in the virtual space searches based on the environment image, and then determines the position based on the search result as the candidate position of the target object to obtain the candidate position information. At the same time, the parameters of the virtual camera corresponding to the camera parameters can also be determined, such as the direction in which the virtual camera should face, the shooting angle, the field of view size, etc.

[0040] S103, based on the preset three-dimensional model, the candidate position information and the parameters of the virtual camera, rendering is performed to generate a rendering image corresponding to the candidate position information; the three-dimensional model and the real space have a second corresponding relationship.

[0041] In this embodiment, the three-dimensional model is a virtual space corresponding to the real space. The three-dimensional model and the real space have a second corresponding relationship. In terms of size, the three-dimensional model and the real space can be one-to-one corresponding, or a reduced version, such as a one-to-ten correspondence. In terms of content, the three-dimensional model can be completely corresponding to the real space, or it can be incompletely corresponding, such as not including the sky, not including moving objects, or only including specific areas, such as the correspondence of the visible range on both sides of the road.

[0042] After the candidate position information and the parameters of the virtual camera are determined, a virtual camera can be set at the candidate position in the preset 3D model, and image rendering is performed based on the parameters of the virtual camera and 3D rendering technology to quickly generate a rendered image corresponding to the candidate position information, such as Figure 3 shown.

[0043] Rendered images include but are not limited to color images, black and white images, contour images, feature point clouds, etc.

[0044] Regarding the construction of a three-dimensional model, in some embodiments, in order to ensure the accuracy of matching the rendered image with the environmental image, dynamic objects, such as moving pedestrians and vehicles, need to be removed when building the three-dimensional model to prevent such information from interfering with the image matching.

[0045] In some embodiments, in order to reduce the storage space of the 3D model and increase the speed of retrieval of the 3D model, dynamic model accuracy can be used. For example, for road scenes with obvious features, the model accuracy corresponding to the road can be reduced. For example, the accuracy of other nearby buildings can be reduced for buildings with unique shapes in the scene; for scenes with small differences and highways without special markings, it is necessary to improve the model accuracy corresponding to the scene and the highway, and record the details of the road and surrounding objects to improve the matching accuracy; for object information that is not time-sensitive, the model accuracy corresponding to the object information is improved, such as road information, street light information, fixed buildings on the roadside, etc.; for object information that is time-sensitive, such as trees (leaves fall in winter) and the sky (clear and cloudy changes during the day and night), the model accuracy corresponding to the object information is reduced.

[0046] In some embodiments, the model accuracy can also be set to be dynamically updated. For example, when the location recognition accuracy of some road sections is very high, it means that the characteristics of the road section are obvious, and the model accuracy corresponding to the road section can be automatically lowered, or a low-precision model (different precision models are supported at the same location) can be automatically called for image rendering; when the recognition accuracy of some road sections is low and the matching result error is large, the model accuracy corresponding to the road section can be increased, or a high-precision model with added color information or other information can be used for image rendering.

[0047] In some embodiments, different modeling modes may be used for different environments, such as using only grid information to record the contour information of an object, or using both grid information and mapping information to record the contour and pattern information of an object.

[0048] In some embodiments, lighting information and the like may also be recorded in the three-dimensional model.

[0049] S104, determining the position information of the target object based on the environment image, the rendered image and the candidate position information.

[0050] In this embodiment, the position information of the target object can be determined by image brute force matching. For example, under a specific positioning accuracy, the rendered images generated by all candidate position information are matched with the real-time environmental image of the target object, and the candidate position information corresponding to the best matching rendered image is found as the position information of the target object in the real space.

[0051] The object positioning method provided in the embodiment of the present application obtains the environment image collected by the target object through its own camera and the camera parameters of the environment image; determines the alternative position information of the target object and the parameters of the virtual camera, and the parameters of the virtual camera have a first corresponding relationship with the camera parameters of the environment image; based on the preset three-dimensional model, the alternative position information and the parameters of the virtual camera, renders and generates a rendered image corresponding to the alternative position information; the three-dimensional model and the real space have a second corresponding relationship; based on the environment image, the rendered image and the alternative position information, the position information of the target object is determined; in this way, in the virtual three-dimensional model matching the real space, a rendered image corresponding to the alternative position information is generated by rendering, and then the rendered image is matched with the environment image actually taken by the target object to obtain the current accurate position information of the target object, which can simply and quickly realize the positioning of the target object, and has lower requirements on the environment in which the target object is located. Compared with obtaining environmental information through various sensors carried on the target object, and then using the prior data collected in advance to calculate real-time positioning information through matching or graph optimization methods, it has higher flexibility and a wider range of applicability. For example, it can be applied to harsh conditions with poor signals such as tunnels and bridge holes.

[0052] In an optional embodiment, if the target object includes multiple cameras set at different positions, each camera can capture an environmental image. In order to improve the speed of matching the environmental image and the virtual image, after determining the candidate position information of the target object, the object positioning method also includes: determining the model parameters corresponding to the candidate position information in the three-dimensional model; based on the model parameters, selecting the environmental image captured by the camera at the target position and the camera parameters for capturing the environmental image, for determining the position information of the target object.

[0053] In this embodiment, the model parameters include but are not limited to model accuracy, model features, and image modes, and the model features include building features, road features, etc. After determining the model parameters corresponding to the alternative position information in the three-dimensional model, the environment image of the target orientation can be selected according to the characteristics of the model parameters, and the parameters of the virtual camera can be determined based on the camera parameters of the environment image of the target orientation, and then the image rendering is performed, and the environment image of the target orientation is matched with the rendered image to determine the position information of the target object. For example, when a vehicle is equipped with multiple cameras in different directions and collects environment images in real time, the real-time environment image of a specific angle can be automatically selected according to the model features of the alternative position, and matched with the rendered image in a specific direction. For example, if the building features in a certain area facing the rear of the vehicle are significant, when positioning the vehicle, the environment image collected by the camera facing the rear of the vehicle can be automatically retrieved, and the environment images captured by other cameras can be ignored. The camera parameters of the environment image facing the rear of the vehicle are used to determine the parameters of the virtual camera, and then the image rendering is performed, and the environment image facing the rear of the vehicle is matched with the rendered image to determine the position information of the vehicle.

[0054] In an optional embodiment, step S104 determines the location information of the target object based on the environment image, the rendered image and the candidate location information, such as Figure 4 As shown, specifically including:

[0055] S1041, comparing the environment image and the rendered image to determine whether the environment image and the rendered image meet a preset condition.

[0056] Specifically, the environment image and the rendered image are compared based on features of the environment image and the rendered image.

[0057] In some embodiments, since the rendered image can be a color picture, a black and white picture, a contour picture or a feature point cloud, and the rendered image is generated under a certain model accuracy corresponding to the alternative position, the accuracy of some objects in the obtained rendered image may not be high, and the environmental image captured by the camera is generally a color picture, in which the accuracy of each object is relatively high. Therefore, in order to more accurately and quickly match the environmental image with the rendered image, before comparing the environmental image with the rendered image, the object positioning method also includes: determining the model parameters corresponding to the alternative position information in the three-dimensional model; and processing the environmental image based on the model parameters.

[0058] When implementing it, Figure 5As shown, after generating the rendered image corresponding to the candidate position information, the model parameters corresponding to the alternative position information are output, that is, the model parameters corresponding to the rendered image. The model parameters include the model accuracy and image mode corresponding to the alternative position information. The image mode includes a black-and-white image, a contour image, or a feature point cloud, etc. Then, the environment image is optimized based on the model parameters, and then the rendered image and the processed environment image are compared to see whether they meet the preset conditions, and then the comparison result is output. For example, if the rendered image is a black-and-white image, and the model accuracy corresponding to the lower right corner of the rendered image is high, then the area in the lower right corner is more sensitive to the difference when the image is compared, and the contribution score is larger when the difference threshold is calculated, then the environment image can be processed into a black-and-white image, and the accuracy of the area in the lower right corner is improved, and the accuracy of other areas of the environment image is reduced, so that when the environment image and the rendered image are compared, more attention is paid to the comparison of the lower right corner area. For another example, the upper half of the rendered image is the sky and the model accuracy is extremely low. When performing image comparison, the threshold of the difference in this part can be zero, and the accuracy of the upper half of the environment image can be adjusted to an extremely low level, so that when the environment image and the rendered image are compared, the difference in the upper half is not compared.

[0059] S1042: When it is determined that the environment image and the rendered image meet a preset condition, determine the candidate position information as the position information of the target object.

[0060] Specifically, the preset condition may include that the features of the environment image and the rendered image are similar and the sizes of the features are similar. For example, if the environment image and the rendered image have common typical buildings and the sizes of the buildings are similar, then the environment image and the rendered image meet the preset condition. If the environment image and the rendered image meet the preset condition, then the candidate position can be determined as the current position of the target object.

[0061] S1043, when it is determined that the environmental image and the rendered image do not satisfy the preset conditions, calculate the position offset information based on the environmental image and the rendered image, update the alternative position information based on the position offset information, and return to the step of obtaining the environmental image captured by the target object through its own camera and the camera parameters for capturing the environmental image, until it is determined that the environmental image and the rendered image satisfy the preset conditions, and then determine that the updated alternative position information is the position information of the target object.

[0062] Specifically, when the rendered image and the environmental image are compared and do not meet the preset conditions but have a certain degree of similarity, the position offset information can be calculated as input for updating the alternative position information to speed up image matching. For example, if the two pictures have a common typical building, but the building in the photographed image is larger, it means that the alternative position information is too far away from the typical building, and the position offset information can be calculated.

[0063] In some embodiments, the position offset information includes the time information of the target object capturing the next frame of the environment image, for example, 50ms. When the alternative position is updated based on the position offset information, the offset distance can be calculated based on the position offset information and the moving speed of the target object, and the alternative position information is updated based on the offset distance. After the alternative position information is updated based on the position offset information, the target object responds to the time information and re-captures the environment image through its own camera to update the environment image. For example, the position offset information can be the time when the vehicle acquires the next frame of the environment image. The vehicle responds to the time and re-captures the environment image through its own camera when the time is up, and regenerates the rendered image based on the updated alternative position information for matching verification.

[0064] Of course, when the rendered image and the environment image do not meet the preset conditions and do not have a certain similarity after comparison, a position offset information can be randomly determined or an alternative position can be reselected to update the alternative position information.

[0065] In this embodiment, the position offset information is set to include the time information of the target object capturing the next frame of the environment image, so that the candidate position information can be quickly updated and the environment image can be updated.

[0066] In this embodiment, the rendering image can be generated by rendering in a virtual space, so the rendering image is generated very quickly. The location information of the target object is determined by comparing the environment image with the rendering image, and further updating the environment image and the rendering image when the environment image and the rendering image do not meet the preset conditions. The speed is fast and the accurate location information of the target object can be obtained.

[0067] In an optional embodiment, although the generation speed of the rendered image is relatively fast, it still takes a certain amount of time to generate the rendered image in real time. In order to improve the speed of image matching and the speed of determining the position information of the target object, in step S104, after determining the position information of the target object based on the environment image, the rendered image and the alternative position information, the object positioning method also includes: obtaining the second moving speed information of the target object; predicting the position information of the target object at the second moment based on the position information of the target object and the second moving speed information as the alternative position information of the second moment, the second moment being the moment after the current moment; rendering and generating a rendered image corresponding to the alternative position information of the second moment based on a preset three-dimensional model, the alternative position information of the second moment and the parameters of the virtual camera, so as to generate a rendered image corresponding to the alternative position information of the second moment in advance.

[0068] For example, based on the vehicle's current position and speed information, multiple possible future location points are predicted as alternative locations, and multiple rendered images are generated in advance, which are then matched with the real-time collected environmental images to improve the matching speed.

[0069] In an optional embodiment, since the real space is constantly changing, it is necessary to dynamically update the three-dimensional space. Therefore, in step S104, after determining the position information of the target object based on the environmental image, the rendered image and the alternative position information, the object positioning method also includes: determining the difference information of the three-dimensional model relative to the real space based on the environmental image and the rendered image; and updating the three-dimensional model based on the difference information.

[0070] For example, based on the environmental images collected by the vehicle in real time and matched to the vehicle's actual position, if specific changes are found in the scene, such as a missing tree or a new building, the corresponding model data can be dynamically generated and added to the three-dimensional model.

[0071] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0072] Figure 6 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0073] like Figure 6 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0074] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0075] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the object positioning method. For example, in some embodiments, the object positioning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the object positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the object positioning method in any other appropriate manner (e.g., by means of firmware).

[0076] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0078] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0080] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0081] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0082] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0084] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for locating an object, comprising: Obtaining an environment image captured by the target object through its own camera and camera parameters for capturing the environment image; Determining candidate position information of the target object and parameters of a virtual camera, wherein the parameters of the virtual camera have a first corresponding relationship with camera parameters of the environment image; Based on the preset three-dimensional model, the candidate position information and the parameters of the virtual camera, rendering and generating a rendering image corresponding to the candidate position information; The three-dimensional model and the real space have a second corresponding relationship; The position information of the target object is determined based on the environment image, the rendered image and the candidate position information.

2. The object positioning method according to claim 1, determining the candidate position information of the target object comprises: Acquire the position information of the target object at a first moment, where the first moment is a moment before the current moment; The candidate position information of the target object is determined based on the position information at the first moment.

3. The object positioning method according to claim 2, wherein determining the candidate position information of the target object based on the position information at the first moment comprises: Acquiring first moving speed information of the target object; Calculating the moving distance of the target object based on the first moving speed information and the time difference information between the current moment and the first moment; Based on the position information at the first moment and the moving distance of the target object, the position information of the target object at the current moment is predicted as candidate position information of the target object.

4. The object positioning method according to claim 1, wherein determining the position information of the target object based on the environment image, the rendered image and the candidate position information comprises: Comparing the environment image with the rendered image; In a case where it is determined that the environment image and the rendered image satisfy a preset condition, determining that the candidate position information is the position information of the target object; When it is determined that the environmental image and the rendered image do not satisfy the preset conditions, position offset information is calculated based on the environmental image and the rendered image, the alternative position information is updated based on the position offset information, and the step of obtaining the environmental image captured by the target object through its own camera and the camera parameters for capturing the environmental image is returned until it is determined that the environmental image and the rendered image satisfy the preset conditions, and then the updated alternative position information is determined to be the position information of the target object.

5. The object positioning method according to claim 4, wherein the position offset information includes time information when the target object captures the next frame of the environment image; Correspondingly, after the candidate position information is updated based on the position offset information, the target object re-collects the environment image through its own camera in response to the time information.

6. The object positioning method according to claim 4, before comparing the environment image with the rendered image, further comprising: Determining model parameters corresponding to the candidate position information in the three-dimensional model; The environment image is processed based on the model parameters.

7. The object positioning method according to claim 1, after determining the position information of the target object based on the environment image, the rendered image and the candidate position information, further comprising: Acquiring second moving speed information of the target object; Predicting the position information of the target object at a second moment based on the position information of the target object and the second moving speed information as candidate position information at the second moment, where the second moment is a moment after the current moment; Based on the preset three-dimensional model, the alternative position information at the second moment and the parameters of the virtual camera, a rendering image corresponding to the alternative position information at the second moment is rendered and generated, so as to generate a rendering image corresponding to the alternative position information at the second moment in advance.

8. The object positioning method according to claim 1, after determining the position information of the target object based on the environment image, the rendered image and the candidate position information, further comprising: Determine, based on the environment image and the rendered image, difference information of the three-dimensional model relative to the real space; The three-dimensional model is updated based on the difference information.

9. The object positioning method according to claim 1, wherein the target object comprises a plurality of cameras arranged at different positions. Accordingly, after determining the candidate position information of the target object, the method further includes: Determining model parameters corresponding to the candidate position information in the three-dimensional model; An environment image captured by a camera at a target orientation and camera parameters for capturing the environment image are selected based on the model parameters, and are used to determine the position information of the target object.

10. An electronic device, comprising: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the object positioning method as described in any one of claims 1-9.

11. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the object positioning method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Positioning method, device and storage medium

    CN107957266A

  • Vehicle positioning method and system

    CN108303721A

  • A method and device for position and posture determination

    CN109003305A

  • Power distribution room field routing inspection method and system based on Hololens mixed reality technology

    CN110246235A

  • Outdoor augmented reality application method based on cross-source image matching

    CN111260794A

Cited By

  • Object positioning method, electronic apparatus, and computer-readable storage medium

    EP4310780A1

  • Object positioning method, electronic apparatus, and computer-readable storage medium

    EP4310780B1