A database construction method, a positioning method and related equipment

Abstract

The present application discloses a positioning method, characterized by being applied to a vehicle and comprising: acquiring a real-time image and information about the natural conditions at the time the real-time image is acquired through an onboard camera; transmitting the real-time image and the natural condition information to a server, and receiving positioning information fed back by the server, the positioning information being obtained by the server based on the real-time image and the natural condition information; and obtaining the vehicle's position information based on the positioning information. The positioning method disclosed in the present application takes into account factors affecting natural conditions and can improve the accuracy of a visual positioning system in positioning a vehicle.

Description

[0001] This application is a divisional case of the Chinese patent application filed with the China Patent Office on June 20, 2018, with application number 201810642562.4 and invention name “A database construction method, a positioning method and related equipment”. Technical Field

[0002] The present application relates to the field of communications, and in particular to a database construction method, a positioning method and related equipment. Background Art

[0003] Vehicle positioning is crucial in autonomous driving technology. During operation, the vehicle's latest position must be acquired in real time to plan subsequent routes. Positioning is typically performed using GPS (global positioning system), cameras, LiDAR, millimeter-wave radar, and / or inertial measurement units (IMUs). However, these methods inevitably suffer from inaccurate positioning accuracy.

[0004] A visual localization method has been proposed. It works by first establishing a database and then identifying and matching scene feature points in the real-time scene with those in the database. The database stores scene keyframes and scene feature points. Keyframes represent real-world images, while scene feature points are extracted from keyframes and belong to keyframes. Scene feature points also have descriptors that describe them, and these descriptors vary depending on the natural environment.

[0005] When the external environment changes, the descriptor information in the collected real-world scene keyframes will also change, causing the corresponding scene feature points to change. When matching scene feature points during the visual positioning process, some scene feature points in the real-world scene keyframes may not match the scene feature points in the database, ultimately leading to inaccurate positioning. Summary of the Invention

[0006] The embodiments of the present application disclose a database construction method, a positioning method and related equipment, which are used to construct a database based on second scene feature point information corresponding to target natural condition information, so that the positioning is more accurate when the database is used for positioning.

[0007] The first aspect of the present application provides a database construction method, comprising:

[0008] After acquiring the image set, the network device determines the target image set that meets the preset overlap requirement. When the mobile device is traveling straight, it acquires image sets at preset distance intervals and then calculates whether the image overlap meets the requirement. When the vehicle is turning, it acquires image sets at preset angle intervals and then calculates whether the image overlap meets the requirement.

[0009] In this embodiment, the image refers to an image of the mobile device and its surrounding environment. The image can be obtained by installing a camera on the mobile device, or the mobile device can have its own image acquisition function, which is not limited here.

[0010] In this embodiment, the target image set includes at least one image, and each image is taken under a unique natural condition, so each image corresponds to a type of natural condition information.

[0011] The natural condition information is determined as follows: the mobile device obtains the location information of the mobile device through the global positioning system GPS, lidar, millimeter wave radar and / or inertial measurement unit IMU, and then sends the location information to the climate server to obtain the natural condition information of the current location.

[0012] The network device analyzes and processes the target image set to obtain scene feature point information. At the same time, each image in the target image set corresponds to a unique natural condition information. Each scene feature point in the scene feature point set is associated with the natural condition information, thereby obtaining a scene feature point information set. The scene feature point information set includes at least one scene feature point information. Each scene feature point information includes the scene feature point's 3D coordinates, pixel coordinates, keyframe ID, and descriptor information. Each descriptor information includes a type of natural condition information. The scene feature point's 3D coordinates, pixel coordinates, and keyframe ID are static indicators, while the descriptor information is a dynamic indicator that changes with the natural condition.

[0013] Visual positioning involves comparing scene feature points to identify identical scene feature points, and then performing positioning based on these identical scene feature points. Static scene feature points are often used for comparison to improve positioning accuracy. To identify representative static scene feature points within a set of scene feature points, a life value calculation is required. The life value represents the probability that a scene feature point is static. A larger life value indicates a greater probability of a static scene feature point, and vice versa.

[0014] The same scene feature point may be photographed by a single mobile device or by multiple mobile devices. First, determine in the scene feature point information set, for a single mobile device, that is, when the scene feature point is observed by one mobile device, the scene feature point whose life value is greater than the first preset life value threshold. Then determine in the first scene feature point information, for multiple mobile devices, that is, when the scene feature point is observed by two or more mobile devices, the scene feature point whose life value is greater than the second preset life value threshold. This scene feature point is the second scene feature point.

[0015] The information of the second scene feature point obtained by filtering in the above manner is compared with the scene feature point information preset in the database. When the second scene feature point information does not match the scene feature point information preset in the database, the second scene feature point information is updated in the database to make the scene feature point information in the database more complete.

[0016] This embodiment has the following advantages: after determining a target image set that meets the preset image overlap requirements, a scene feature point information set is determined based on the target image set and the natural condition information corresponding to each image in the graphic set, and then second scene feature point information corresponding to scene feature points in the scene feature information set whose life value is greater than a first preset life value threshold for a single mobile device and greater than a second preset life value threshold for multiple mobile devices is determined. When the second scene feature point information does not match the scene feature point information preset in the database, a database is constructed based on the second scene feature point information. In this embodiment, after the second scene feature point information is screened and obtained in the above manner, if the second scene feature point information related to a certain natural condition information does not exist in the database, the database is constructed based on the second scene feature point information, so that when the constructed database is used for positioning, the positioning is more accurate.

[0017] Based on the first aspect, in a first implementable manner of the first aspect, the second scene feature point information not matching the scene feature point information preset in the database specifically includes: the second scene feature point information does not exist in the database.

[0018] When the second scene feature point does not exist in the database, it is obvious that the second scene feature point information cannot exist in the database. At this time, the second scene feature point information is stored in the database. The stored second scene feature point information includes the 3D coordinates, pixel coordinates, key frame ID and descriptor information of the second scene feature point.

[0019] In this embodiment, a case of constructing a database according to the second scene feature point information is described, which increases the feasibility of the solution.

[0020] Based on the first aspect, in a second achievable method of the first aspect, the second scene feature point information does not match the scene feature point information preset in the database, specifically including: the second scene feature point information exists in the database, but the second scene feature point information does not include target descriptor information about the target natural condition information.

[0021] When the database contains second scene feature point information, but the second scene feature point information stored in the database is different from the second scene feature point information determined from the image, the difference is due to changes in the dynamic indicators of the two, namely, the descriptor information. For example, the second scene feature point information in the database is the 3D coordinates, pixel coordinates, key frame ID, and descriptor 1 information of the second scene feature point. The second scene feature point information determined in the image is the 3D coordinates, pixel coordinates, key frame ID, and descriptor 2 information of the second scene feature point.

[0022] At this time, it is necessary to add target descriptor information related to the target natural condition information to the second scene feature point in the database, that is, the information of descriptor 2.

[0023] In this embodiment, another case of constructing a database according to the second scene feature point information is described, which increases the flexibility of the solution implementation.

[0024] Based on the first or second implementation of the first aspect, in a third implementation of the first aspect, before adding the second scene feature point information to the database, the method further includes:

[0025] Determining the 3D coordinates of the second scene feature point in the second scene feature point information, where the 3D coordinates are determined synchronously when determining the positioning information of the image to which the second scene feature point belongs;

[0026] Calculate the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point, and determine whether the differences are all greater than a first preset threshold value;

[0027] If so, determining that the second scene feature point information does not exist in the database;

[0028] If the difference between the 3D coordinates of any preset scene feature point in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point exists in the database, and the second scene feature point information also exists.

[0029] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, which increases the feasibility of the solution.

[0030] Based on the second implementation of the first aspect, in a fourth implementation of the first aspect, before adding the target descriptor information to the second scene feature point information preset in the database, the method further includes:

[0031] When the second scene feature point information exists in the database, determining at least one descriptor of the second scene feature point information preset in the database, and determining whether there is a descriptor among the at least one descriptor whose distance from a descriptor corresponding to the target descriptor information in the image is less than a preset distance threshold;

[0032] If the distance between the descriptor preset in the database and the descriptor corresponding to the target descriptor information in the second scene feature point information is greater than the second preset distance threshold, it is determined that the second scene feature point information preset in the database does not include the target descriptor information.

[0033] In this embodiment, a method for determining whether target descriptor information exists in a database is described, thereby increasing the feasibility of the solution.

[0034] Based on the first aspect and the first to second implementations of the first aspect, in a fifth implementation of the first aspect, the life value of the scene feature point on a single mobile device is f, and the calculation formula of f is: Wherein, n represents the number of times a certain scene feature point is observed in a single mobile device. A single mobile device can conduct multiple experiments to acquire images, so a certain scene feature point may be observed multiple times by a single mobile device. n0 is the average number of times any scene feature point is observed by at least one mobile device obtained in advance through model training, and σ is the variance of the number of times any scene feature point is observed by at least one mobile device obtained in advance through model training.

[0035] In this embodiment, a formula for calculating the life value of a scene feature point on a single mobile device is described, thereby increasing the feasibility of the solution.

[0036] Based on the fifth implementation of the first aspect, in a sixth implementation of the first aspect, a certain scene feature point may be photographed by multiple mobile devices, and the life value of the scene feature point on the multiple mobile devices is F, and the calculation formula of F is: f is the health value of the scene feature point on a single mobile device, and B is the weight coefficient corresponding to each of the multiple mobile devices, with each mobile device corresponding to a weight coefficient. For example, if a scene feature point is captured by three mobile devices, each of the three mobile devices has a corresponding weight coefficient. The health value of the scene feature point on a single mobile device is multiplied by the weight coefficient of the mobile device, and the sum of the three mobile devices is then used to obtain the health value of the scene feature point when observed by multiple mobile devices.

[0037] In this embodiment, a plurality of mobile devices refers to at least two mobile devices.

[0038] In this embodiment, the formula for calculating the life value of scene feature points on multiple mobile devices is described, which increases the feasibility of the solution.

[0039] Based on the sixth implementation of the first aspect, in the seventh implementation of the first aspect, β i The calculation formula is: i =γ t +γ g +γ c , the γ t is the temporal continuity indicator of the scene feature point observed on multiple mobile devices, the γ g is the geometric continuity index of the scene feature points observed on multiple mobile devices, the γ c It is a description consistency indicator of the scene feature points observed on multiple mobile devices.

[0040] Among them, γ t The geometric continuity index γ is related to the time interval between different mobile devices observing the same scene feature points. g It is related to the Euclidean distance between the feature points of the same scene observed by different mobile devices, describing the consistency γ c It is related to the description distance between feature points of the same scene observed by different mobile devices.

[0041] In this embodiment, the weight coefficient β i The calculation formula is explained to increase the feasibility of the solution.

[0042] Based on the first aspect and the first to second implementations of the first aspect, in an eighth implementation of the first aspect, determining the target image set that meets the preset image overlap requirement includes:

[0043] When the mobile device moves straight, it first moves at a preset distance d k+1Select an image, and when the difference between the overlap of the selected image and the preset overlap threshold is within a preset accuracy range, determine that the selected image is a target image, and then select images according to the preset distance interval to obtain a target image set.

[0044] When the difference between the selected image's coincidence degree and the preset coincidence degree threshold is not within the preset accuracy range, the distance interval d k+1 The difference between the overlap of the corresponding image and the preset overlap threshold is calculated to obtain the distance interval d. k+1 How much distance interval should be increased or decreased based on the selected image to make the selected image meet the preset coincidence requirement, so as to obtain the distance interval d of the next selected image k+2 , and then d k+2 Repeat the above steps as the distance interval of the selected image. k+2 Select an image that meets the overlap requirement, then follow d k+2 Select an image to get a target image set.

[0045] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection can be avoided. At the same time, the defect of excessive data volume when matching with existing scene feature point information in the database can also be avoided.

[0046] Based on the eighth implementation of the first aspect, in the ninth implementation of the first aspect, the preset distance interval d k+1 ;

[0047] The calculation formula of the preset distance interval is: k+1 =d k +d k (α * -α), wherein the α * is the preset overlap threshold, the d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of coincidence between images calculated when selecting images.

[0048] In this embodiment, the calculation formula of the preset distance interval is explained to increase the completeness of the solution.

[0049] Based on the first aspect and the first to second implementations of the first aspect, in a tenth implementation of the first aspect, determining the target image set that meets the preset image overlap requirement includes:

[0050] When the mobile device bends, it first moves at a preset angle interval θ k+1An image is selected. When the difference between the overlap degree of the selected image and the preset overlap degree threshold is within a preset accuracy range, the selected image is determined to be a target image. Then, images are repeatedly selected according to the preset angle interval to obtain a target image set.

[0051] When the difference between the selected image overlap and the preset overlap threshold is not within the preset accuracy range, the image overlap is calculated based on the angle interval θ. k+1 The difference between the overlap degree of the corresponding image and the preset overlap degree threshold is calculated and the angle interval θ is required. k+1 Based on this, how much angular interval should be increased or decreased to select images so that the selected images meet the preset coincidence requirements, thereby obtaining the angular interval θ for the next image selection. k+2 , and then θ k+2 Repeat the above steps as the angle interval of the selected image. k+2 Select an image that meets the overlap requirement, then follow θ k+2 Select an image to get a target image set.

[0052] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection can be avoided. At the same time, the defect of excessive data volume when matching with existing scene feature point information in the database can also be avoided.

[0053] Based on the tenth implementation of the first aspect, in the eleventh implementation of the first aspect, the preset angle interval θ k+1 .

[0054] The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ k The angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of coincidence between images calculated when selecting images.

[0055] In this embodiment, the calculation formula of the preset angle interval is explained to increase the completeness of the solution.

[0056] Based on the first aspect and the first to second implementations of the first aspect, in a twelfth implementation of the first aspect, the scene feature point information includes descriptor information corresponding to the natural condition information, and obtaining the scene feature point information set based on the target image set and the natural condition information includes:

[0057] 1> The network device processes a target image to obtain scene feature points of the target image;

[0058] 2> Establish a correspondence between the scene feature point, the target image to which the scene feature point belongs (or the key frame ID to which the scene feature point belongs), and the natural condition information corresponding to the target image, thereby forming the scene feature point information, which also includes information such as the 3D coordinates and pixel coordinates of the scene feature point.

[0059] Among them, the scene feature point may belong to multiple target images, that is, the scene feature point is included in multiple target images. The natural condition information corresponding to each target image is generally different. Therefore, there can be multiple descriptor information in the scene feature point information. At the same time, it is not ruled out that the natural condition information corresponding to two or more target images is the same.

[0060] Repeat steps 1> and 2>, and perform the above processing on each image until the scene feature point information set is obtained.

[0061] In this embodiment, the method for determining the scene feature point information set is described to increase the completeness of the solution.

[0062] Based on the first aspect and the first to second implementations of the first aspect, in a thirteenth implementation of the first aspect, after constructing the database according to the second scene feature point information, the method further includes:

[0063] Determine the scene feature point information in the database after the construction is completed, the scene feature point information being the third scene feature point information;

[0064] When the feature quantity control score FNCS of the third scene feature point corresponding to the third scene feature point information is less than a preset FNCS threshold, the third scene feature point information is deleted from the database. The size of FNCS can indicate the probability of the scene feature point being used during positioning and the number of descriptors contained in the scene feature point information.

[0065] In this embodiment, the preset FNCS may be determined in advance through multiple experiments.

[0066] In this embodiment, the management of scene feature points after the database is built is described. Scene feature points with low FNCS values can be deleted to facilitate database management.

[0067] Based on the thirteenth implementation of the first aspect, in the fourteenth implementation of the first aspect, the calculation formula of the feature quantity control score FNCS is: described is the probability that the scene feature point is used in positioning, M is the total number of positioning at the location of the scene feature point, m i Indicates the number of times the scene feature point is used during positioning. It is the ratio of the number of descriptors of the scene feature point to the total number of descriptors in the image to which the scene feature point belongs.

[0068] In this embodiment, the calculation formula of the feature quantity control score FNCS is explained, which increases the feasibility of the solution.

[0069] A second aspect of the embodiments of the present application provides a positioning method, characterized by comprising:

[0070] The network device obtains a real-time image of the mobile device. In this embodiment, the real-time image refers to the image of the mobile device and its surrounding environment. The real-time image can be obtained by installing a camera on the mobile device. The mobile device can also have its own image acquisition function, which is not limited here.

[0071] After the network device obtains the real-time image, it analyzes and processes the real-time image to determine at least one first descriptor information. The first descriptor information includes the target natural condition information of the location of the mobile device when the mobile device or the external camera of the mobile device shoots the real-time image. The real-time image contains at least one feature point information. The natural condition information when the real-time image is shot is certain, so one feature point information only includes one descriptor information, and then the at least one descriptor information of the real-time image contains the same target natural condition information.

[0072] The target natural condition information can be determined by a network device, or determined by a mobile device and then sent to the network device. The target natural condition information is determined as follows: first, the network device or mobile device determines the location information of the mobile device when the real-time image is taken, which can be determined by the global positioning system GPS, lidar, millimeter wave radar and / or inertial measurement unit IMU. The network device or mobile device then determines the target natural condition information based on the location information.

[0073] Comparing the descriptor information preset in the database with the at least one first descriptor information to determine identical descriptor information, wherein the comparison is specifically performed as follows:

[0074] Determine a descriptor information of at least one descriptor information;

[0075] Determine whether the distance between the descriptor corresponding to the descriptor information and the descriptor corresponding to any descriptor information preset in the database is less than a preset distance threshold;

[0076] If so, it is determined that the descriptor and the descriptor in the database whose distance is less than the preset distance threshold are the same descriptor, and the corresponding descriptor information is also the same; if not, the two are different descriptors, and the corresponding descriptor information is also different.

[0077] Repeat the above steps to compare each descriptor in the at least one descriptor information with the descriptors preset in the database to determine whether the same descriptor information is obtained.

[0078] In this embodiment, the descriptor information preset in the database is obtained after the database is constructed. After the network device determines the target image set that meets the preset image coincidence requirements, it obtains a scene feature point information set based on the target image set and the natural condition information corresponding to each image in the target image set. The first scene feature point information corresponding to the first scene feature point that meets the preset life value requirements is selected from the scene feature point information set, and the second descriptor information corresponding to the target natural condition information in the first scene feature point information is determined. When the second descriptor information does not match the descriptor information preset in the database, the database is constructed based on the second descriptor information.

[0079] In this embodiment, the specific process of constructing the database is similar to the process of constructing the database in the first aspect of this application, and will not be repeated here.

[0080] After determining that the same descriptor information is obtained, visual positioning is performed on the real-time image according to the same descriptor information.

[0081] In this embodiment, the positioning process after the database is built is described to increase the practicality of the solution.

[0082] Based on the second aspect, in a first implementation manner of the second aspect, positioning the real-time image using the same descriptor information includes:

[0083] After determining the descriptor information in the real-time image that is identical to that in the database, determine the first scene feature point information in the database to which the identical descriptor information belongs, search the database to obtain the 3D coordinates, pixel coordinates and other information of the first scene feature point, and then combine the first scene feature point information with the positioning calculation formula to obtain the position of the target mobile device when shooting the real-time image.

[0084] In this embodiment, a specific method of real-time image positioning is described to increase the feasibility of the solution.

[0085] Based on the first implementation of the second aspect, in the second implementation of the second aspect, the positioning calculation formula is:

[0086] The location of the target mobile device when the real-time image is captured wherein is the pixel coordinate of the first scene feature point in the real-time image, π C is the intrinsic parameter matrix of the camera, the π C For converting 3D coordinates to pixel coordinates, the is the scene feature point in the database The pose of the image relative to the world coordinate system, is the pixel coordinate of the first scene feature point in the database, the value of i ranges from 1 to n, n is a positive integer, and the first scene feature point corresponds to the first scene feature point information.

[0087] There are n scene feature points in the real-time image that are matched with scene feature points in the database. By (π C ) -1 Convert to get the 3D coordinate relative to the car, and then pass Convert to get 3D coordinates relative to the world coordinate system, and then pass Convert the pixel coordinates relative to the database. is the scene feature point in the database The pose of the key frame relative to the world coordinate system. The pixel coordinates obtained by conversion are the pixel coordinates of the scene feature points that match the database. The reprojection error model is obtained by subtracting the two. Finally, the value of the reprojection error model is minimized by the optimization method to obtain the real-time posture of the car.

[0088] In this embodiment, the specific algorithm of the positioning formula is described to increase the completeness of the solution.

[0089] Based on the second aspect and the first to second implementations of the second aspect, in a third implementation of the second aspect, after determining at least one first descriptor information according to the real-time image, the method further includes:

[0090] Comparing the descriptor information preset in the database with the at least one first descriptor information to determine different descriptor information;

[0091] In this embodiment, the method of comparing descriptor information is similar to the method of comparing descriptor information described in the second aspect above, and the details are not repeated here.

[0092] If there is descriptor information in the real-time image that is different from that in the database, the descriptor information of the different descriptors will be added to the database so that the positioning will be more accurate when the database is used for positioning.

[0093] In this embodiment, during the positioning process, the database can be updated according to different descriptor information, so that the database is more complete, thereby making the positioning more accurate.

[0094] In the third implementation manner based on the second aspect, in the fourth implementation manner of the second aspect, when the second scene feature point information to which the different descriptor information belongs does not exist in the database, constructing the database according to the different descriptor information includes:

[0095] When the second scene feature point does not exist in the database, the second scene feature point information also does not exist. The second scene feature point belongs to a different descriptor. In this case, the second scene feature point information containing different descriptor information needs to be added to the database. In this embodiment, there may be multiple different descriptor information. Since one second scene feature point information of a real-time image can only contain one different descriptor, the number of second scene feature point information that needs to be added to the database is also multiple.

[0096] In this embodiment, a case of constructing a database according to different descriptor information is described, which increases the practicality of the solution.

[0097] In the third implementation manner based on the second aspect, in the fifth implementation manner of the second aspect, when the second scene feature point information to which the different descriptor information belongs exists in the database, constructing the database according to the different descriptor information includes:

[0098] When a second scene feature point exists in the database, second scene feature point information also exists. However, the second scene feature point information in the database is different from the second scene feature point information in the real-time image, that is, the second scene feature point information does not contain the different descriptor information. In this case, the different descriptor information needs to be added to the second scene feature point information in the database. For example, the second scene feature point information in the database is the 3D coordinates, pixel coordinates, key frame ID, and descriptor 1 information of the second scene feature point. The second scene feature point information determined in the real-time image is the 3D coordinates, pixel coordinates, key frame ID, and descriptor 2 information of the second scene feature point. In this case, the descriptor 2 information needs to be added to the second scene feature point information in the database.

[0099] In this embodiment, another case of constructing a database according to different descriptor information is described, which increases the completeness of the solution.

[0100] Based on the third implementation manner of the second aspect, in a sixth implementation manner of the second aspect, before constructing the database according to the different descriptor information, the method further includes:

[0101] Determining the 3D coordinates of the second scene feature point included in the second scene feature point information, where the 3D coordinates are determined synchronously when determining the positioning information of the image to which the second scene feature point belongs;

[0102] Calculate whether the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold value;

[0103] If so, determining that the second scene feature point information does not exist in the database;

[0104] If the difference between the 3D coordinates of any preset scene feature point in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point exists in the database, and the second scene feature point information also exists.

[0105] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, thereby increasing the feasibility of the solution.

[0106] The third aspect of the present application provides a database, which is deployed on a server;

[0107] The database is constructed by second scene feature point information that does not match the preset scene feature point information in the database, and the second scene feature point information is the scene feature point information corresponding to the scene feature point in the first scene feature point information where the life value of multiple mobile devices is greater than the second preset life value threshold, and the first scene feature point information is the scene feature point information corresponding to the scene feature point in the scene feature point information set where the life value of a single mobile device is greater than the first preset life value threshold, the scene feature point information set is obtained based on the target image set and the natural condition information corresponding to each image, and the scene feature point set includes at least one scene feature point information, the target image set includes at least one image that meets the preset image overlap requirement, and each image corresponds to one natural condition information.

[0108] The process of forming the database in this embodiment is similar to the process of building the database in the first aspect, and will not be described in detail here.

[0109] In this embodiment, the method of constructing a database is described. Once the database is constructed, it can be used for visual positioning to make positioning more accurate.

[0110] Based on the third aspect, in a first implementation of the third aspect, the server further includes a processor;

[0111] The second scene feature point information does not match the scene feature point information preset in the database, including: the second scene feature point information does not exist in the database;

[0112] The database is constructed by the second scene feature point information and includes:

[0113] When the second scene feature point information does not exist in the database, the second scene feature point information is added to the database, so the database is formed by the server adding the second scene feature point information to the database, and the second scene feature point information includes target descriptor information about target natural condition information.

[0114] In this embodiment, a method of constructing a database of feature points of the second scene is described, which increases the feasibility of the solution.

[0115] Based on the third aspect, in a second implementation of the third aspect, the server further includes a processor;

[0116] The second scene feature point information does not match the scene feature point information preset in the database, including: the second scene feature point information exists in the database, and the second scene feature point information does not include target descriptor information about target natural condition information;

[0117] The database is constructed by the second scene feature point information and includes:

[0118] When the scene feature point information preset in the database is the same as the second scene feature point information, but the descriptor information of the two is different, the different descriptor information is added to the database, so the database is formed by the server adding the target descriptor information to the second scene feature point information preset in the database.

[0119] In this embodiment, another method of constructing a database of the second scene feature points is described, which increases the diversity of solution implementation.

[0120] A fourth aspect of the present application provides a network device, including:

[0121] a determining unit, configured to determine a target image set that meets a preset image coincidence requirement, wherein the target image set includes at least one image, and each image corresponds to a type of natural condition information;

[0122] a processing unit, configured to obtain a scene feature point information set based on the target image set and the natural condition information corresponding to each image, wherein the scene feature point set includes at least one scene feature point information;

[0123] The determining unit is further configured to determine, in the scene feature point information set, first scene feature point information corresponding to a scene feature point having a life value greater than a first preset life value threshold for a single mobile device, wherein the size of the life value is used to indicate a probability that the scene feature point is a static scene feature point;

[0124] The determining unit is further configured to determine, in the first scene feature point information, second scene feature point information corresponding to a scene feature point where the health value of the plurality of mobile devices is greater than a second preset health value threshold;

[0125] A database construction unit is configured to construct the database according to the second scene feature point information when the second scene feature point information does not match the scene feature point information preset in the database.

[0126] In this embodiment, after the second scene feature point information is obtained by screening in the above manner, when the second scene feature point information related to a certain natural condition information does not exist in the database, the database is constructed based on the second scene feature point information, so that when the constructed database is used for positioning, the positioning is more accurate.

[0127] Based on the fourth aspect, in a first implementable manner of the fourth aspect, the second scene feature point information does not match the scene feature point information preset in the database, including:

[0128] The second scene feature point information does not exist in the database;

[0129] The database construction unit is specifically configured to add the second scene feature point information to the database, where the second scene feature point information includes target descriptor information about target natural condition information.

[0130] In this embodiment, a case of constructing a database according to the second scene feature point information is described, which increases the feasibility of the solution.

[0131] Based on the fourth aspect, in a second implementable manner of the fourth aspect, the second scene feature point information does not match the scene feature point information preset in the database, including:

[0132] The second scene feature point information exists in the database, and the second scene feature point information does not include target descriptor information about target natural condition information;

[0133] The database construction unit is specifically configured to add the target descriptor information to the second scene feature point information preset in the database.

[0134] In this embodiment, another case of constructing a database according to the second scene feature point information is described, which increases the flexibility of the solution implementation.

[0135] Based on the first or second implementable manner of the fourth aspect, in a third implementable manner of the fourth aspect, the determining unit is further configured to determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0136] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0137] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0138] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, which increases the feasibility of the solution.

[0139] Based on the second implementation manner of the fourth aspect, in the fourth implementation manner of the fourth aspect, the determining unit is further used to determine at least one descriptor information of the second scene feature point preset in the database;

[0140] The network device further includes:

[0141] A judging unit, configured to judge whether, in the at least one descriptor information, there is a descriptor corresponding to a descriptor information whose distance from the descriptor corresponding to the target descriptor information is less than a preset distance threshold;

[0142] The determining unit is further configured to determine that the second scene feature point information preset in the database does not include target descriptor information if, in the at least one descriptor information, there is no descriptor corresponding to any descriptor information whose distance to the descriptor corresponding to the target descriptor information is less than a preset distance threshold.

[0143] In this embodiment, a method for determining whether target descriptor information exists in a database is described, thereby increasing the feasibility of the solution.

[0144] Based on the fourth aspect and the first and second possible implementations of the fourth aspect, in a fifth possible implementation of the fourth aspect, the life value of the scene feature point on a single mobile device is f, and the calculation formula of f is:

[0145] Wherein, n represents the number of times the scene feature point is observed in a single mobile device, n0 is the average number of times the preset scene feature point is observed, and σ is the variance of the number of times the preset scene feature point is observed.

[0146] In this embodiment, a formula for calculating the life value of a scene feature point on a single mobile device is described, thereby increasing the feasibility of the solution.

[0147] Based on the fifth possible implementation of the fourth aspect, in a sixth possible implementation of the fourth aspect, the life value of the scene feature point on the plurality of mobile devices is F, and the calculation formula of F is: f is the life value of the scene feature point on a single mobile device, B is a weight coefficient corresponding to each mobile device, and one weight coefficient corresponds to each mobile device among the multiple mobile devices.

[0148] In this embodiment, the formula for calculating the life value of scene feature points on multiple mobile devices is described, which increases the feasibility of the solution.

[0149] Based on the sixth possible implementation of the fourth aspect, in a seventh possible implementation of the fourth aspect, the β i The calculation formula is: i =γ t +γ g +γ c , the γ t is the temporal continuity indicator of the scene feature point observed on multiple mobile devices, the γ g is the geometric continuity index of the scene feature points observed on multiple mobile devices, the γ c It is a description consistency indicator of the scene feature points observed on multiple mobile devices.

[0150] In this embodiment, the weight coefficient β i The calculation formula is explained to increase the feasibility of the solution.

[0151] Based on the fourth aspect and the first and second possible implementations of the fourth aspect, in an eighth possible implementation of the fourth aspect, the determination unit is specifically configured to select images according to a preset distance interval.

[0152] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0153] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection and the defect of excessive data volume when subsequently matching with existing scene feature point information in the database can be avoided.

[0154] Based on the eighth possible implementation of the fourth aspect, in a ninth possible implementation of the fourth aspect, the preset distance interval d k+1 ;

[0155] The calculation formula of the preset distance interval is: k+1 =d k +d k (α*-α), wherein α* is the preset overlap threshold, and d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of overlap between images when selecting images.

[0156] In this embodiment, the calculation formula of the preset distance interval is explained to increase the completeness of the solution.

[0157] Based on the fourth aspect and the first and second possible implementations of the fourth aspect, in a tenth possible implementation of the fourth aspect, the determining unit is specifically configured to select images according to preset angle intervals;

[0158] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0159] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection and the defect of excessive data volume when subsequently matching with existing scene feature point information in the database can be avoided.

[0160] Based on the tenth possible implementation of the fourth aspect, in an eleventh possible implementation of the fourth aspect, the preset angle interval θ k+1 ;

[0161] The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ kThe angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of overlap between images when selecting images.

[0162] In this embodiment, the calculation formula of the preset angle interval is explained to increase the completeness of the solution.

[0163] Based on the fourth aspect and the first and second possible implementations of the fourth aspect, in a twelfth possible implementation of the fourth aspect, the scene feature point information includes descriptor information corresponding to the natural condition information, and the processing unit is specifically configured to: 1> process one of the target images to obtain scene feature points;

[0164] 2> forming the scene feature point information using the scene feature point, the target image to which the scene feature point belongs, and the natural condition information corresponding to the target image;

[0165] Repeat steps 1> and 2> until the scene feature point information set is obtained.

[0166] In this embodiment, the method for determining the scene feature point information set is described to increase the completeness of the solution.

[0167] Based on the fourth aspect and the first and second possible implementations of the fourth aspect, in a thirteenth possible implementation of the fourth aspect, the determining unit is further configured to determine the third scene feature point information in the database after the construction is completed;

[0168] The database construction unit is further configured to delete the third scene feature point information from the database when a feature quantity control score FNCS of the third scene feature point corresponding to the third scene feature point information is less than a preset FNCS threshold.

[0169] In this embodiment, the management of scene feature points after the database is built is described. Scene feature points with low FNCS values can be deleted to facilitate database management.

[0170] Based on the thirteenth possible implementation of the fourth aspect, in the fourteenth possible implementation of the fourth aspect, the calculation formula of the feature quantity control score FNCS is: described is the probability that the scene feature point is used in positioning, It is the ratio of the number of descriptors of the scene feature point to the total number of descriptors in the image to which the scene feature point belongs.

[0171] In this embodiment, the calculation formula of the feature quantity control score FNCS is explained, which increases the feasibility of the solution.

[0172] A fifth aspect of the present application provides a positioning method, which is applied to a visual positioning system, and includes:

[0173] An acquisition unit, used for acquiring real-time images;

[0174] a determining unit, configured to determine at least one first descriptor information based on the real-time image, wherein the first descriptor information includes information about target natural conditions when the real-time image is captured;

[0175] The determining unit is further configured to compare descriptor information preset in a database with the at least one first descriptor information to determine identical descriptor information, wherein the descriptor information preset in the database is determined by the network device as a target image set that meets a preset image coincidence requirement, and then obtain a scene feature point information set based on the target image set and the natural condition information corresponding to each image in the target image set, select first scene feature point information corresponding to a first scene feature point that meets a preset life value requirement from the scene feature point information set, and then construct the database based on second descriptor information corresponding to the target natural condition information in the first scene feature point information, wherein the second descriptor information does not match the descriptor information preset in the database, and the scene feature point information includes descriptor information corresponding to the natural condition information;

[0176] A positioning unit is used to position the real-time image using the same descriptor information.

[0177] In this embodiment, the positioning process after the database is built is described to increase the practicality of the solution.

[0178] Based on the fifth aspect, in a first possible implementation of the fifth aspect, the positioning unit is specifically configured to determine first scene feature point information corresponding to the same descriptor information in a database;

[0179] The position of the target mobile device when the real-time image is captured is calculated based on the first scene feature point information and a positioning calculation formula.

[0180] In this embodiment, a specific method of real-time image positioning is described, which increases the feasibility and practicality of the solution.

[0181] Based on the first implementable manner of the fifth aspect, in a second implementable manner of the fifth aspect, the positioning calculation formula is:

[0182] The location of the target mobile device when the real-time image is captured wherein is the pixel coordinate of the first scene feature point in the real-time image, π C is the intrinsic parameter matrix of the camera, the π C For converting 3D coordinates to pixel coordinates, the is the scene feature point in the database The pose of the image relative to the world coordinate system, is the pixel coordinate of the first scene feature point in the database, the value of i ranges from 1 to n, n is a positive integer, and the first scene feature point corresponds to the first scene feature point information.

[0183] In this embodiment, the specific algorithm of the positioning formula is described to increase the completeness of the solution.

[0184] Based on the fifth aspect and the first to second possible implementations of the fifth aspect, in a third possible implementation of the fifth aspect, the determining unit is further configured to compare the descriptor information preset in the database with the at least one first descriptor information to determine different descriptor information;

[0185] The network device further includes a database construction unit;

[0186] The database construction unit is specifically configured to construct the database according to the different descriptor information.

[0187] In this embodiment, during the positioning process, the database can be updated according to different descriptor information, so that the database is more complete, thereby making the positioning more accurate.

[0188] Based on the third implementable method of the fifth aspect, in the fourth implementable method of the fifth aspect, when the second scene feature point information to which the different descriptor information belongs does not exist in the database, the database construction unit is specifically used to add the second scene feature point information containing the different descriptor information to the database.

[0189] In this embodiment, a case of constructing a database according to different descriptor information is described, which increases the practicality of the solution.

[0190] Based on the third implementable method of the fifth aspect, in the fifth implementable method of the fifth aspect, when the second scene feature point information to which the different descriptor information belongs exists in the database, the construction unit is specifically used to add the different descriptor information to the second scene feature point information in the database.

[0191] In this embodiment, another case of constructing a database according to different descriptor information is described, which increases the completeness of the solution.

[0192] Based on the third implementation manner of the fifth aspect, in a sixth implementation manner of the fifth aspect, the determining unit is further configured to determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0193] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0194] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0195] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, thereby increasing the feasibility of the solution.

[0196] A sixth aspect of the present application provides a network device, characterized in that the network device includes: a memory, a transceiver, a processor, and a bus system;

[0197] Wherein, the memory is used to store programs;

[0198] The processor is configured to execute the program in the memory, comprising the following steps:

[0199] Determining a target image set that meets a preset image coincidence requirement, wherein the target image set includes at least one image, and each image corresponds to a type of natural condition information;

[0200] Obtaining a set of scene feature point information according to the target image set and the natural condition information corresponding to each image, wherein the scene feature point set includes at least one scene feature point information;

[0201] Determining first scene feature point information corresponding to a scene feature point in the scene feature point information set, where the life value of a single mobile device is greater than a first preset life value threshold, where the size of the life value is used to indicate a probability that the scene feature point is a static scene feature point;

[0202] Determining second scene feature point information corresponding to scene feature points in which the health values of multiple mobile devices are greater than a second preset health value threshold in the first scene feature point information;

[0203] When the second scene feature point information does not match the scene feature point information preset in the database, constructing the database according to the second scene feature point information;

[0204] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0205] In this embodiment, after the second scene feature point information is obtained by screening in the above manner, when the second scene feature point information related to a certain natural condition information does not exist in the database, the database is constructed based on the second scene feature point information, so that when the constructed database is used for positioning, the positioning is more accurate.

[0206] Based on the sixth aspect, in a first implementable manner of the sixth aspect, the second scene feature point information does not match the scene feature point information preset in the database, including:

[0207] The second scene feature point information does not exist in the database;

[0208] The processor is specifically configured to:

[0209] The second scene feature point information is added to the database, where the second scene feature point information includes target descriptor information about target natural condition information.

[0210] In this embodiment, a case of constructing a database according to the second scene feature point information is described, which increases the feasibility of the solution.

[0211] Based on the sixth aspect, in a second implementable manner of the sixth aspect, the second scene feature point information does not match the scene feature point information preset in the database, including:

[0212] The second scene feature point information exists in the database, and the second scene feature point information does not include target descriptor information about target natural condition information;

[0213] The processor is specifically configured to:

[0214] The target descriptor information is added to the second scene feature point information preset in the database.

[0215] In this embodiment, another case of constructing a database according to the second scene feature point information is described, which increases the flexibility of the solution implementation.

[0216] Based on the first or second implementable manner of the sixth aspect, in a third implementable manner of the sixth aspect, the determining unit and the processor are further configured to:

[0217] determining the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0218] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0219] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0220] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, which increases the feasibility of the solution.

[0221] Based on the second implementation manner of the sixth aspect, in a fourth implementation manner of the sixth aspect, the determining unit and the processor are further configured to:

[0222] Determining at least one descriptor information of the second scene feature point preset in a database;

[0223] Determine whether, in the at least one descriptor information, a descriptor corresponding to a descriptor information and a descriptor corresponding to the target descriptor information have a distance less than a preset distance threshold;

[0224] If not, it is determined that the second scene feature point information preset in the database does not include target descriptor information.

[0225] In this embodiment, a method for determining whether target descriptor information exists in a database is described, thereby increasing the feasibility of the solution.

[0226] Based on the sixth aspect and the first and second possible implementations of the sixth aspect, in a fifth possible implementation of the sixth aspect, the life value of the scene feature point on a single mobile device is f, and the calculation formula for f is:

[0227] Wherein, n represents the number of times the scene feature point is observed in a single mobile device, n0 is the average number of times the preset scene feature point is observed, and σ is the variance of the number of times the preset scene feature point is observed.

[0228] In this embodiment, a formula for calculating the life value of a scene feature point on a single mobile device is described, thereby increasing the feasibility of the solution.

[0229] Based on the fifth possible implementation of the sixth aspect, in a sixth possible implementation of the sixth aspect, the life value of the scene feature point on multiple mobile devices is F, and the calculation formula of F is: f is the life value of the scene feature point on a single mobile device, B is a weight coefficient corresponding to each mobile device, and one weight coefficient corresponds to each mobile device among the multiple mobile devices.

[0230] In this embodiment, the formula for calculating the life value of scene feature points on multiple mobile devices is described, which increases the feasibility of the solution.

[0231] Based on the sixth possible implementation of the sixth aspect, in a seventh possible implementation of the sixth aspect, the β i The calculation formula is: i =γ t +γ g +γ c , the γ t is the temporal continuity indicator of the scene feature point observed on multiple mobile devices, the γ g is the geometric continuity index of the scene feature points observed on multiple mobile devices, the γ c It is a description consistency indicator of the scene feature points observed on multiple mobile devices.

[0232] In this embodiment, the weight coefficient β i The calculation formula is explained to increase the feasibility of the solution.

[0233] Based on the sixth aspect and the first and second possible implementations of the sixth aspect, in an eighth possible implementation of the sixth aspect, the processor is specifically configured to:

[0234] Select images according to preset distance intervals;

[0235] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0236] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection and the defect of excessive data volume when subsequently matching with existing scene feature point information in the database can be avoided.

[0237] Based on the eighth possible implementation of the sixth aspect, in a ninth possible implementation of the sixth aspect, the preset distance interval d k+1 ;

[0238] The calculation formula of the preset distance interval is: k+1 =d k +d k (α * -α), wherein the α *is the preset overlap threshold, the d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of overlap between images when selecting images.

[0239] In this embodiment, the calculation formula of the preset distance interval is explained to increase the completeness of the solution.

[0240] Based on the sixth aspect and the first and second possible implementations of the sixth aspect, in a tenth possible implementation of the sixth aspect, the processor is specifically configured to:

[0241] Select images according to preset angle intervals;

[0242] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0243] In this embodiment, by selecting images that meet the overlap requirements, the problem of excessive data volume when processing images due to blind image selection and the defect of excessive data volume when subsequently matching with existing scene feature point information in the database can be avoided.

[0244] Based on the tenth possible implementation of the sixth aspect, in an eleventh possible implementation of the sixth aspect, the preset angle interval θ k+1 ;

[0245] The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ k The angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of overlap between images when selecting images.

[0246] In this embodiment, the calculation formula of the preset angle interval is explained to increase the completeness of the solution.

[0247] Based on the sixth aspect and the first and second implementable manners of the sixth aspect, in a twelfth implementable manner of the sixth aspect, the scene feature point information includes descriptor information corresponding to the natural condition information, and the processor is specifically configured to:

[0248] 1> Processing a target image to obtain scene feature points;

[0249] 2> forming the scene feature point information using the scene feature point, the target image to which the scene feature point belongs, and the natural condition information corresponding to the target image;

[0250] Repeat steps 1> and 2> until the scene feature point information set is obtained.

[0251] In this embodiment, the method for determining the scene feature point information set is described to increase the completeness of the solution.

[0252] Based on the sixth aspect and the first and second possible implementations of the sixth aspect, in a thirteenth possible implementation of the sixth aspect, the processor is further configured to:

[0253] Determining the feature point information of the third scene in the database after the construction is completed;

[0254] When the feature quantity control score FNCS of the third scene feature point corresponding to the third scene feature point information is less than a preset FNCS threshold, the third scene feature point information is deleted from the database.

[0255] In this embodiment, the management of scene feature points after the database is built is described. Scene feature points with low FNCS values can be deleted to facilitate database management.

[0256] Based on the thirteenth possible implementation of the sixth aspect, in the fourteenth possible implementation of the sixth aspect, a calculation formula for the feature quantity control score FNCS is: described is the probability that the scene feature point is used in positioning, It is the ratio of the number of descriptors of the scene feature point to the total number of descriptors in the image to which the scene feature point belongs.

[0257] In this embodiment, the calculation formula of the feature quantity control score FNCS is explained, which increases the feasibility of the solution.

[0258] A seventh aspect of the present application provides a network device, the network device belonging to a visual positioning system, the network device comprising: a memory, a transceiver, a processor, and a bus system;

[0259] The transceiver is used to obtain real-time images;

[0260] Wherein, the memory is used to store programs;

[0261] The processor is configured to execute the program in the memory, comprising the following steps:

[0262] determining at least one first descriptor information according to the real-time image, wherein the first descriptor information includes information about the natural conditions of the target when the real-time image is captured;

[0263] Comparing descriptor information preset in a database with the at least one first descriptor information to determine identical descriptor information, wherein the network device determines a target image set that satisfies a preset image coincidence requirement for the descriptor information preset in the database, obtaining a scene feature point information set based on the target image set and natural condition information corresponding to each image in the target image set, selecting first scene feature point information corresponding to a first scene feature point that satisfies a preset life value requirement from the scene feature point information set, and then constructing the database based on second descriptor information corresponding to the target natural condition information in the first scene feature point information, wherein the second descriptor information does not match the descriptor information preset in the database, and the scene feature point information includes descriptor information corresponding to the natural condition information;

[0264] Positioning the real-time image using the same descriptor information;

[0265] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0266] In this embodiment, the positioning process after the database is built is described to increase the practicality of the solution.

[0267] Based on the seventh aspect, in a first possible implementation of the seventh aspect, the processor is specifically configured to:

[0268] Determining first scene feature point information corresponding to the same descriptor information in a database;

[0269] The position of the target mobile device when the real-time image is captured is calculated based on the first scene feature point information and a positioning calculation formula.

[0270] In this embodiment, a specific method for positioning the real-time image is described, which increases the feasibility of the solution.

[0271] Based on the first implementable manner of the seventh aspect, in a second implementable manner of the seventh aspect, the positioning calculation formula is:

[0272] The location of the target mobile device when the real-time image is captured wherein is the pixel coordinate of the first scene feature point in the real-time image, π C is the intrinsic parameter matrix of the camera, the πC For converting 3D coordinates to pixel coordinates, the is the scene feature point in the database The pose of the image relative to the world coordinate system, is the pixel coordinate of the first scene feature point in the database, the value of i ranges from 1 to n, n is a positive integer, and the first scene feature point corresponds to the first scene feature point information.

[0273] In this embodiment, the specific algorithm of the positioning formula is described to increase the completeness of the solution.

[0274] Based on the seventh aspect and the first to second possible implementations of the seventh aspect, in a third possible implementation of the seventh aspect, the processor is further configured to:

[0275] Comparing the descriptor information preset in the database with the at least one first descriptor information to determine different descriptor information;

[0276] The database is constructed according to the different descriptor information.

[0277] In this embodiment, during the positioning process, the database can be updated according to different descriptor information, so that the database is more complete, thereby making the positioning more accurate.

[0278] Based on the third implementable manner of the seventh aspect, in a fourth implementable manner of the seventh aspect, when the second scene feature point information to which the different descriptor information belongs does not exist in the database, the processor is specifically configured to:

[0279] The second scene feature point information including the different descriptor information is added to the database.

[0280] In this embodiment, a case of constructing a database according to different descriptor information is described, which increases the practicality of the solution.

[0281] Based on the third implementable manner of the seventh aspect, in a fifth implementable manner of the seventh aspect, when the second scene feature point information to which the different descriptor information belongs exists in the database, the processor is specifically configured to:

[0282] The different descriptor information is added to the second scene feature point information in the database.

[0283] In this embodiment, another case of constructing a database according to different descriptor information is described, which increases the completeness of the solution.

[0284] Based on the third implementation manner of the seventh aspect, in a sixth implementation manner of the seventh aspect, the processor is further configured to:

[0285] determining the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0286] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0287] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0288] In this embodiment, a method for determining whether the second scene feature point exists in the database is described, thereby increasing the feasibility of the solution.

[0289] An eighth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, enable the computer to execute the methods described in the above aspects.

[0290] A ninth aspect of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the methods described in the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0291] Figure 1 This is a schematic diagram of the relationship between data structure and data type in the image database of this application;

[0292] Figure 2 This is a schematic diagram of an embodiment of the present application when applied to a vehicle;

[0293] Figure 3 A structural diagram of the visual positioning system of this application;

[0294] Figure 4 This is another structural diagram of the visual positioning system of this application;

[0295] Figure 5 A schematic diagram of an embodiment of the database construction method of the present application;

[0296] Figure 6 A schematic diagram of an embodiment of selecting a target image set for this application;

[0297] Figure 7 Schematic diagram of an embodiment of selecting feature point information of the second scene for this application;

[0298] Figure 8 This is a diagram showing the relationship between the life value of the scene feature point and the number of times the scene feature point is observed;

[0299] Figure 9 This is a schematic diagram of another embodiment of the database construction method of the present application;

[0300] Figure 10 This is a schematic diagram of an embodiment of the present application for determining whether a scene feature point exists in a database;

[0301] Figure 11 This is a schematic diagram of an embodiment of the positioning method of the present application;

[0302] FIG12( a ) shows a case where the scene feature point information in the image of the present application does not match the scene feature point information preset in the database;

[0303] FIG12( b ) shows another case where the scene feature point information in the image of the present application does not match the scene feature point information preset in the database;

[0304] Figure 13 This is a schematic diagram of another embodiment of the positioning method of the present application;

[0305] Figure 14 A structural diagram of the network device of this application;

[0306] Figure 15 Another structural diagram of the network device of this application;

[0307] Figure 16 This is another structural diagram of the network device of this application. DETAILED DESCRIPTION

[0308] In the embodiments of this application, Figure 1 As shown, the database stores scene keyframe information, scene feature point information, and descriptor information, and there is an association between the three. Scene keyframe information includes image, position, and posture. A scene keyframe has at least one scene feature point. The scene feature point information includes the ID information of the scene keyframe to which the scene feature point belongs, pixel coordinates, 3D coordinates, and descriptor information. A scene feature point has at least one descriptor. Part of the descriptor information is the scene feature point descriptor ξ in the traditional visual field, and the other part is the natural condition attribute E of the scene when the scene feature point was collected. When the natural condition attribute E changes, the descriptor ξ will also change. For example, the descriptor of the same image taken on a cloudy day and a sunny day is different. The pixel coordinates, 3D coordinates, and the ID information of the scene keyframe to which the scene feature point belongs are static attributes of the scene feature point and will not change due to changes in the external environment.

[0309] In the embodiment of the present application, the description sub-information is different under different natural conditions. Different natural conditions refer to different viewing directions, different weather and / or different lighting conditions. Different natural conditions may also be other situations, which are not specifically limited here.

[0310] The embodiments of this application are primarily used in visual positioning systems. The principle of visual positioning is to compare scene feature points in a captured image with scene feature points in a database. If the scene feature points in the captured image match the corresponding scene feature points in the database, they are considered to be the same scene feature points. The 3D coordinates of the scene feature points that match the database are then used for positioning. This application can be applied to the positioning of mobile devices such as drones, V2X vehicles, and mobile phones during movement.

[0311] like Figure 2 As shown, taking the real-time positioning of a vehicle during operation as an example: first, the real-time image information of vehicle A during driving and the positioning information obtained through non-visual positioning methods such as GPS are determined. Vehicle A then sends the real-time image to the server. Vehicle A can also send positioning information to the server. After receiving the real-time image, the server determines the natural condition information of the location. Alternatively, vehicle A determines the natural condition information based on the positioning information and sends the natural condition information to the server. The server then finds multiple descriptors related to the natural condition information in the real-time image and compares the determined multiple descriptors with pre-stored descriptors in the database. The database belongs to the server and is used to store scene key frames and scene feature point information.

[0312] When the descriptors in the database are identical to those in the real-time image, the descriptors that are successfully matched in the database are found. The successful descriptor matching proves that the scene feature points to which the descriptors belong are the same scene feature points. The 3D coordinates of the same scene feature points are found, and the vehicle can use the 3D coordinates of the same scene feature points for positioning. The descriptors in the real-time image may be exactly the same as the descriptors in the database. In this case, the 3D coordinates of the scene feature points to which the exactly same descriptors belong are directly used for positioning. If only some of the descriptors in the real-time image can find corresponding descriptors in the database, the 3D coordinates of the scene feature points to which the partially identical descriptors belong are first used for positioning. After positioning is completed, the information of the different descriptors is obtained, and then the information of the different descriptors is updated to the database to optimize the database, so that the positioning using the optimized database is more accurate in subsequent positioning.

[0313] Before achieving positioning in this application scenario, a database construction process must be performed. This process uses a large number of images as a foundation. Scene feature points in the images are selected using a vital point algorithm to obtain a large amount of scene feature point information. This information is then compared with existing scene feature point information in the database, and the database is updated and optimized, with any scene feature point information not already in the database being added. The vital point algorithm accurately selects representative scene feature points, making the database more accurate for visual positioning.

[0314] Figure 3 The figure shows a possible structure of a visual positioning system, in which a locator is used to obtain the positioning information of a mobile device and, optionally, its posture information; an image acquirer is used to capture images of the mobile device; and the mobile device receives images and positioning information from the image acquirer and the locator, and then transmits them to a network device. The network device can also directly obtain images and positioning information without using the mobile device, that is, directly connect to the image acquirer and the locator, which are not limited here. After receiving the image, the network device compares scene feature points to achieve positioning and can also update and manage its own database.

[0315] Optionally, one possible scenario is that the mobile device sends positioning information to the network device. Another possible scenario is that the mobile device determines the natural condition information based on the positioning information and sends the natural condition information to the network device without sending the positioning information. The specific situation is not limited here.

[0316] based on Figure 3 The visual positioning system shown in Figure 1 is a possible physical structure. Figure 4 As shown:

[0317] Specific locators include: Global Positioning Systems, cameras, LiDAR, millimeter-wave radar, and inertial measurement units. IMUs can obtain positioning information and the attitude of the mobile device. Locators can be components of the mobile device or external devices connected to it, and the specifics are not limited here.

[0318] Mobile devices can specifically include vehicles, mobile phones, and drones.

[0319] The image acquirer may specifically be a camera, a component of a mobile device, or an external device connected to the mobile device, which is not specifically limited here.

[0320] The network device can be a cloud server or a mobile device with data processing capabilities, which is not limited here. Figure 1The data model for visual positioning shown in FIG. 4 introduces the relationship between scene key frames, scene feature points, and descriptors.

[0321] Based on the structure of the above network device and mobile device, the embodiment of the present application proposes a database construction method and a positioning method. The present application includes two parts. One part is the construction process of the database on the network device side, the purpose of which is to make the database better used for visual positioning through reasonable management of the database. The other part is the process of performing visual positioning after the database construction is completed. The two parts will be introduced separately below, and the database construction process is as follows: Figure 5 As shown:

[0322] 501. Determine a target image set that meets a preset image overlap requirement.

[0323] The network device obtains data information. In this embodiment, the data information may be image information, location information, posture information, or natural condition information, which is not limited here.

[0324] The network device may obtain image information of the mobile device while it is in motion by installing a camera on the mobile device to obtain images captured by the camera. The mobile device may also have an image capture function, and the network device then obtains images captured by the mobile device. During operation, the mobile device captures images at a predetermined interval. The captured images primarily represent image information of the surrounding environment of the mobile device during movement. The selected interval is manually set and may be 0.01s or 0.001s, etc., and is not limited to this specific interval.

[0325] The image information includes at least one image. The posture and real-time location information of the mobile device are different when each image is taken. The posture indicates the driving angle and direction of the mobile device. The real-time location information of the mobile device can be obtained through the global positioning system GPS, lidar, millimeter wave radar and / or inertial measurement unit IMU.

[0326] In this embodiment, each image has a one-to-one correspondence with the position of the mobile device and the posture of the mobile device when the image was captured, as shown in Table 1 below:

[0327] Table 1

[0328]

[0329] After obtaining the image information, a target image set that meets the preset overlap requirement is selected based on the data information. Optionally, the process for determining the target image set may include: the mobile device may first filter the acquired images according to the preset overlap requirement, and then send the filter results to the network device. The target image filtering process may also be performed by the network device, that is, the network device obtains the images and filters them to obtain the target image set. The specific process is not limited here.

[0330] It should be noted that the basis for determining the target image set is different when the mobile device is traveling straight and curved. When the car is traveling straight, it is necessary to determine the target image set that meets the requirements at a certain distance interval. When the car is traveling curved, it is necessary to determine the target image set that meets the requirements at a certain angle interval. The specific steps are as follows: Figure 6 As shown:

[0331] A. According to the distance interval (or angle interval) d k Select an image.

[0332] A predefined distance interval or angle interval is used to determine the expected image selection. For example, on a straight road, an image is acquired every 1 meter the car travels. Or on a curved road, an image is acquired every 5-degree change in the car's driving angle.

[0333] B. Determine the current distance interval (or angle interval) d k The overlap α between the following images.

[0334] After the image selection is completed, the overlap between two adjacent images in the selected images is calculated. The calculation formula is:

[0335] The overlap between images α = the number of scene feature points that are the same between the current image and the adjacent image n old / Number of different scene feature points between the current image and the adjacent images n new The number of scene feature points n in the current image total ,and

[0336] n total =n old +n new .

[0337] C. Determine whether the difference between the overlap of the selected image and the preset overlap threshold is within the preset accuracy range.

[0338] The calculation formula is: |α * -α|<Δ α ;

[0339] Among them, α * is the preset coincidence threshold, α * Generally take 1, Δ αis the preset accuracy value, Δ α The value range of α is 0.1 to 0.2. * and Δ α Other values can also be taken, which are not limited here. The preset accuracy range is 0- to Δ α .

[0340] D. If not, recalculate the distance interval (or angle interval).

[0341] If the difference between the overlap of the selected image and the preset overlap threshold is not within the preset accuracy range, the distance interval of the selected image is redefined. First, determine whether the distance interval (or angle interval) Δd needs to be increased. k .

[0342] When the mobile device moves straight, Δd k =d k (α * -α);

[0343] Among them, d k That is the distance interval of the last selected image, d k and α * and α have been obtained in the above steps.

[0344] Get the new distance interval d for scene key frame selection k+1 =d k +Δd k .

[0345] At the same time, d k+1 Determine the distance interval for obtaining the scene key frame image again, return to step A and re-execute the above process until a distance interval d is obtained. k+n , when selecting images according to this distance interval, the overlap degree meets the preset conditions.

[0346] E. If yes, determine that the selected image belongs to the target image set.

[0347] If the difference between the selected image's coincidence degree and the preset coincidence degree threshold is within the preset accuracy range, then the distance interval d k The selected image is the target image, according to d k Select multiple target images to obtain a target image set.

[0348] When the mobile device bends, the angle interval that meets the overlap requirement is first determined, and then the images are filtered. The specific process is similar to the method of filtering images according to the distance interval, and the details are not repeated here.

[0349] 502. Obtain a scene feature point information set based on the target image set and the natural condition information corresponding to each image.

[0350] The network device processes the target image set to obtain scene feature points. Scene feature points can be considered as pixels in the target image with significantly different grayscale values from other pixels. The network device then determines the natural condition information at that location based on the location of the mobile device when each target image was captured. A correspondence between the scene feature points and the natural condition information is established to obtain scene feature point information. It is understood that the scene feature point information includes not only the natural condition information but also the 3D coordinates, pixel coordinates, and descriptor information of the scene feature point.

[0351] In this embodiment, multiple images may include the same scene feature point, so the correspondence between the scene feature point and the natural condition information can be one-to-one, or one scene feature point can correspond to multiple natural condition information. The descriptor information changes with the change of the natural condition information, so one scene feature point information may include multiple descriptor information.

[0352] For example, there are target image 1 and target image 2 in the target image set. Target image 1 is taken on a sunny day with a light intensity of 400lx, and target image 2 is taken on a cloudy day with a light intensity of 300lx. There are scene feature points 1 and scene feature points 2 in target image 1, and scene feature points 2 and scene feature points 3 in target image 2. The target image set is parsed to obtain scene feature points 1, scene feature points 2, and scene feature points 3. There is 1 descriptor in the scene feature points, which corresponds to the natural condition information of target image 1. There are two descriptors in scene feature points 2, which correspond to the natural condition information of target image 1 and the natural condition information of target image 2 respectively. There is 1 descriptor in scene feature point 3, which corresponds to the natural condition information of target image 2.

[0353] After obtaining the set of scene feature points, representative scene feature points are selected. It can be understood that positioning is more accurate when using stationary objects for positioning. Therefore, representative scene feature points can be scene feature points related to stationary objects such as signboards, road signs and buildings on the road. Specifically, the method of selecting representative scene feature points is to select them based on the life value of the scene feature points. The size of the life value can represent the probability that the scene feature point is a static scene feature point. The larger the life value, the greater the probability that the scene feature point is a static scene feature point. First, the life value of the scene feature point is calculated from the perspective of a single vehicle end, and the scene feature points are screened once. Secondly, since a scene feature point is generally observed by multiple mobile devices, the scene feature points obtained from the first screening need to be screened once to calculate the life value from the perspective of multiple mobile devices for a second screening. Multiple mobile devices refer to at least two mobile devices. The specific embodiment is shown in steps 503 and 504 below:

[0354] 503. Determine first scene feature point information corresponding to a scene feature point in which the life value of a single mobile device is greater than a first preset life value threshold in the scene feature point information set.

[0355] Please refer to Figure 7 ,The screening process of scene feature points includes:

[0356] A. Calculate the first life value of the scene feature point observed on a single vehicle side based on the preset model relationship.

[0357] First, the model is trained to obtain Figure 8 The relationship between the number of times a scene feature point is observed and its life value is shown. It can be seen that a normal curve forms between the number of times a scene feature point is observed, n, and its life value, f. From the perspective of a single mobile device, if a scene feature point appears in only a few frames, it is likely noise and needs to be discarded. If it appears in a relatively large number of frames, it is likely an image of another mobile device moving synchronously with the mobile device in this embodiment and also needs to be discarded.

[0358] according to Figure 8 Determine the average value n0 and variance σ of the number of times the scene feature point is observed. Then calculate the first life value of each scene feature point in the scene feature point set. The calculation formula is:

[0359]

[0360] Here, n represents the number of times a scene feature point in the scene feature point set is observed in a single vehicle end.

[0361] B. Determine whether the first health value is greater than a first preset threshold value.

[0362] When the first life value of a scene feature point is greater than the first preset threshold, the scene feature point is the first scene feature point, and then it is determined from the perspective of multiple mobile devices whether the first scene feature point after one screening meets the life value requirement.

[0363] If the first life value of a scene feature point is less than or equal to the first preset threshold, it means that the first life value of the scene feature point is too low, and the scene feature point is discarded.

[0364] 504. Determine second scene feature point information corresponding to a scene feature point in which the health values of the plurality of mobile devices are greater than a second preset health value threshold in the first scene feature point information.

[0365] C. Calculate the second life value of scene feature points when they are observed by multiple mobile devices.

[0366] Determining whether the first scene feature point acquired by multiple mobile devices is the same scene feature point can be determined based on the 3D coordinates or pixel coordinates of the scene feature point, or other methods can be used to determine whether the scene feature points acquired by multiple mobile devices are the same scene feature point, the specific details of which are not limited herein. For example, among the scene feature points acquired by multiple mobile devices, scene feature points having the same 3D coordinates or having a 3D coordinate difference within a preset difference range are considered the same scene feature point.

[0367] The second life value of the first scene feature point observed by multiple mobile devices is calculated using the following formula:

[0368]

[0369] f is the life value of the scene feature point on a single mobile device, and B is the weight coefficient corresponding to each mobile device. For a scene feature point on multiple mobile devices, the weight coefficient corresponding to each mobile device is generally different. Weight coefficient β i The calculation formula is: i =γ t +γ g +γ c , γ t is the temporal continuity indicator of scene feature points observed on multiple mobile devices, γ g is the geometric continuity index of scene feature points observed on multiple mobile devices, γ c It is a description consistency indicator of scene feature points observed on multiple mobile devices.

[0370] in,

[0371] Δ t is the time interval between different mobile devices observing the same scene feature point, Δ1 and Δ2 are preset values. It can be seen that the time interval between different mobile devices observing the same scene feature point and its γ t Negatively correlated.

[0372] γ g and γ c The calculation process of γ t It is similar and will not be described in detail here. It should be noted that in calculating the geometric continuity index γ g When Δ is defined as the Euclidean distance between the feature points of the same scene observed by different mobile devices. c When , Δ is defined as the description distance between the feature points of the same scene observed by different mobile devices.

[0373] D. Determine whether the second health value is greater than a second preset threshold.

[0374] Determine the first scene feature point whose second life value is greater than or equal to the second preset threshold as the second scene feature point, the second scene feature point is a representative mature scene feature point, and prepare to add the information of the second scene feature point to the database.

[0375] If the life value of the first scene feature point is less than the second preset life value threshold, the first scene feature point is discarded.

[0376] 505. When the second scene feature point information does not match the scene feature point information preset in the database, construct a database according to the second scene feature point information.

[0377] After determining the second scene feature point information to be added to the database, it is also necessary to determine whether the second scene feature point information already exists in the database to avoid repeated addition of scene feature point information to the database.

[0378] The second scene feature point information is compared with scene feature point information preset in the database. If the scene feature point information preset in the database does not match the second scene feature point information, a database is constructed according to the second scene feature point information.

[0379] In this embodiment, after the second scene feature point information is obtained by screening in the above manner, when the second scene feature point information related to a certain natural condition information does not exist in the database, the database is constructed based on the second scene feature point information, so that when the constructed database is used for positioning, the positioning is more accurate.

[0380] In this embodiment, a scene feature point information includes the 3D coordinates, pixel coordinates, descriptor information related to natural conditions and the key frame ID of the scene feature point, wherein the 3D coordinates, pixel coordinates and the key frame ID of the scene feature point represent the static indicators of the scene feature point, which are generally fixed, while the descriptor information is a dynamic indicator that changes with changes in natural conditions. Therefore, the scene feature point information preset in the database does not match the second scene feature point information. The possible situation is that the second scene feature point information does not exist in the database, or the second scene feature point information exists in the database, but the second scene feature point information in the database is different from the descriptor information contained in the second scene feature point information determined in the image. Please refer to Figure 9 , which will be explained below.

[0381] 901. Determine a target image set that meets a preset image overlap requirement.

[0382] 902. Obtain a scene feature point information set based on the target image set and the natural condition information corresponding to each image.

[0383] 903. Determine first scene feature point information corresponding to a scene feature point in which the life value of a single mobile device is greater than a first preset life value threshold in the scene feature point information set.

[0384] 904. Determine second scene feature point information corresponding to a scene feature point in which the health value of the plurality of mobile devices is greater than a second preset health value threshold in the first scene feature point information.

[0385] 905. When the second scene feature point information does not exist in the database, add the second scene feature point information to the database.

[0386] It is determined whether the second scene feature point exists in the database. If not, the corresponding second scene feature point information does not exist in the database, and the information of the second scene feature point is added to the database.

[0387] Please refer to Figure 10 , determining whether the second scene feature point exists in the database specifically includes the following steps:

[0388] A. Determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information.

[0389] First, determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information. Since the scene feature point may be observed by multiple mobile devices, first obtain the 3D coordinates of the scene feature point observed by multiple mobile devices when observing a scene feature point from multiple mobile devices, and then calculate the average of the multiple 3D coordinates. and standard deviation σ, and then the 3D coordinates of the scene feature points measured by each vehicle are compared with the mean Comparing the two, when the Euclidean distance between them is greater than 3σ, it means that the 3D coordinate error measured by the vehicle is large, and the 3D coordinate is deleted. Then the remaining 3D coordinates are used to recalculate the mean of the 3D coordinates of the scene feature points. and standard deviation σ, and judge the 3D coordinates and mean Is the Euclidean distance of less than 3σ? Repeat this process until all remaining 3D coordinates are equal to the mean. The Euclidean distance between them is less than 3σ, and the coordinate mean at this time is output. The 3D coordinates of the scene feature points.

[0390] For example, the number of vehicle terminals is N, and the 3D coordinates of the same scene feature points observed by the N vehicle terminals are 3D1, 3D2, 3D3 and 3Dn respectively.

[0391] First calculate

[0392] judge and …and

[0393] If so, then That is, the 3D coordinates of the scene feature points. If at least one of the 3D coordinates (for example, 3D1) is If the Euclidean distance between them is greater than 3σ, then delete 3D1 and take the average of 3D2, 3D3 to 3Dn to get It should be noted that 3σ is a fixed value preset by the system, and the specific value of 3σ is not limited here.

[0394] B. Determine whether the second scene feature point exists in the database based on the 3D coordinates of the second scene feature point.

[0395] After calculating the 3D coordinates of the scene feature points, compare them with the 3D coordinates of any scene feature point in the image database. d , it is determined that it belongs to the same scene feature point as the scene feature point in the database. When the 3D coordinates of each scene feature point preset in the database are compared with the 3D coordinates of the scene feature point, and the Euclidean distance is greater than the first preset threshold value, the scene feature point is determined to be a new scene feature point, and the new scene feature point information (i.e., the second scene feature point information) is added to the database. d The specific value of is not limited here. At the same time, in this embodiment, the number of new scene feature points is also not limited.

[0396] The new scene feature point information added to the database includes: the pixel coordinates, 3D coordinates, key frame ID and target descriptor information of the new scene feature point. The new scene feature point information may also include other descriptor information in addition to the target descriptor information, which is not limited here.

[0397] As shown in FIG12( a ), for example, scene feature point 4 does not exist in the database for the image, and thus, descriptor 10 for scene feature point 4 does not exist either. Descriptor 10 is a different descriptor in this embodiment. In this case, information about a second scene feature point containing a different descriptor, namely, information about scene feature point 4, is added to the database.

[0398] or,

[0399] 906. When the second scene feature point information exists in the database and the second scene feature point information does not include target descriptor information about target natural condition information, add the target descriptor information to the second scene feature point information preset in the database.

[0400] After A and B determine whether the second scene feature point exists in the database according to step 905 of the above embodiment, if the second scene feature point information exists in the database, it is determined whether the second scene feature point contains target descriptor information related to the target natural condition information. Specifically:

[0401] C. Determine at least one descriptor information of a second scene feature point preset in the database.

[0402] When the second scene feature point exists in the database, since the second scene feature point has different descriptors under different natural condition information, the second scene feature point information preset in the database includes at least one descriptor information.

[0403] D. Determine whether, in the at least one descriptor information, there is a descriptor corresponding to a descriptor information whose distance from the descriptor corresponding to the target descriptor information is less than a preset distance threshold.

[0404] It is determined whether there is a descriptor corresponding to the descriptor information in the at least one descriptor information in the database, and the distance between the descriptor corresponding to the target descriptor information of the second scene feature point determined in the image and the descriptor is less than a preset distance threshold.

[0405] In this embodiment, the target descriptor information is descriptor information about the target's natural condition information. One possible scenario is that the descriptor corresponding to the target descriptor information is the descriptor with the smallest sum of distances to all other descriptors among all the descriptors of the scene feature point. Another possible scenario is that the descriptor corresponding to the target descriptor information is any descriptor among all the descriptors of the scene feature point.

[0406] When the distance between the target descriptor and at least one descriptor in the database is greater than the preset distance threshold, the target descriptor is judged to be a new descriptor. When the distance between the target descriptor and a descriptor in the database is less than or equal to the preset distance threshold, they are judged to be the same descriptor.

[0407] If the target descriptor is a new descriptor, the information of the target descriptor is stored in the database. If it is the same descriptor, no update processing is performed. In this embodiment, the number of new descriptors is not limited here.

[0408] As shown in Figure 12(b), for example, the database includes scene feature points 1, 2, and 3 that are identical to the target image, but scene feature point 3 in the database does not contain the target descriptor for scene feature point 3 in the image, namely, descriptor 9. In this case, the descriptor sub-information corresponding to the different descriptor 9 is added to the database.

[0409] In this embodiment, two situations in which scene feature points in an image match scene feature points in a database are described. While updating and improving the database information, only the parts that are different from the scene feature point information in the database are updated, thereby avoiding a large increase in the amount of data in the database and facilitating database management.

[0410] Furthermore, after the scene feature points or descriptors are updated, in order to optimize the database and make it easier to manage, some scene feature points can be deleted through screening conditions, and only scene feature points that meet the requirements are retained. The retention of scene feature points follows two principles:

[0411] a. Keep as many scene feature points as possible that are frequently used in positioning. This is expressed using the MNI indicator. The calculation formula for MNI is:

[0412]

[0413] M is the total number of positioning at the location of the scene feature point, m i Indicates the number of times the scene feature point is used during positioning.

[0414] b. Retain scene feature points with less dynamic descriptor information as much as possible. This is represented by the indicator FNI, which is calculated as follows:

[0415]

[0416] It is the ratio of the number of descriptors in a scene feature point to the total number of descriptors in the scene key frame to which the scene feature point belongs.

[0417] Combining the above two criteria, the calculation formula for the feature quantity control score FNCS is obtained as follows:

[0418]

[0419] The larger the FNCS value of a scene feature point, the more frequently it is used and the fewer descriptors it has, indicating that it is more robust to changes in natural conditions. Therefore, scene feature points with low FNCS scores need to be deleted when managing scene feature points.

[0420] The above describes the process of building the database. Please refer to the following Figure 11 ,The following will explain how the database is used during positioning and how the database is ,updated synchronously during positioning.

[0421] 1101. Acquire real-time images.

[0422] While the target mobile device is in motion, real-time image information of the target mobile device's movement is captured by installing a camera on the target mobile device, or by installing cameras at regular intervals along the road. It is understood that the captured real-time image is a picture of the road and environment surrounding the vehicle as it moves.

[0423] After the camera captures the real-time image, it can be sent directly to the network device or sent to the network device via the target mobile device, which is not limited to the specific one. At the same time, the target mobile device itself can also have an image acquisition function.

[0424] 1102. Determine at least one first descriptor information according to the real-time image.

[0425] The network device processes the real-time image to obtain at least one first descriptor information, which includes the target natural condition information when the real-time image is taken. The target natural condition information can be determined by the network device or determined by the mobile device and then sent to the network device. The specific details are not limited here.

[0426] The target natural condition information when capturing real-time images is determined based on the real-time positioning information of the mobile device. The real-time positioning information of the mobile device can be obtained through a global positioning system (GPS), a lidar and / or millimeter-wave radar, or an inertial measurement unit (IMU), the specifics of which are not limited herein. After obtaining the real-time positioning information, the natural condition information at that location is determined to be the target natural condition information.

[0427] 1103. Compare the descriptor information preset in the database with the at least one first descriptor information to determine identical descriptor information.

[0428] The same scene is imaged differently under different viewing angles, weather conditions, and lighting conditions. For example, the pixels around the corner of a road sign on a clear day are significantly different from the pixels around the same corner on a cloudy day. Another example is that the pixels around the corner of the front of the sign are significantly different from the pixels around the corner of the back of the sign. Thus, the descriptors of the corners of the sign at the same location vary greatly under different weather conditions, lighting conditions, and viewing angles. Therefore, it can be understood that the real-time image captured under the natural conditions corresponding to the target natural condition information has a unique natural condition. Therefore, a scene feature point in the real-time image contains only one descriptor information. However, a real-time image has multiple scene feature points, so there is at least one descriptor information in the real-time image that includes the target natural condition information. One possible scenario is that there are N first descriptors in the real-time image, and M of the N first descriptors include the target natural condition information, where N and M are both positive integers and M is less than or equal to N.

[0429] Each first descriptor in the at least one first descriptor is compared with pre-set descriptor information in the database to identify identical descriptors. For example, if a real-time image includes descriptors 1, 2, ..., and N, and the descriptors in the database are compared, it is found that descriptors 1, 5, ..., and N-1 are identical in the database.

[0430] The method of judging whether the descriptor is the same as the descriptor in the database is similar to the method of judging whether the descriptors are the same when the database is constructed, that is, judging based on the distance between the descriptors, which will not be described in detail here.

[0431] In this embodiment, the descriptor information preset in the database is obtained by constructing the database according to steps 501 to 505 of the embodiment. The details will not be repeated here.

[0432] 1104. Use the same descriptor information to locate the real-time image.

[0433] First, determine that the same descriptor is obtained, search for scene feature points to which the same descriptor belongs in the database, find the 3D coordinates of these scene feature points, and then use the 3D coordinates of these scene feature points for positioning.

[0434] For example, in Figure 12(a), when comparing the real-time image with the database, we find that both descriptor 1 and descriptor 4 have corresponding descriptors in the database. Therefore, we determine that scene feature point 1 belonging to descriptor 1 is the same as scene feature point 1 belonging to descriptor 1 in the database. At the same time, scene feature point 4 belonging to descriptor 4 is the same as scene feature point 4 belonging to descriptor 4 in the database. We then find the 3D coordinates of the same scene feature points: 3D coordinate 1 and 3D coordinate 2. 3D coordinate 1 and 3D coordinate 2 are then used for positioning calculations.

[0435] The network devices use the same scene feature point information for positioning calculation. After determining the same scene feature points in the database, the mobile device posture is obtained according to the preset algorithm. In this embodiment, the positioning calculation formula is:

[0436]

[0437] is the pose of the mobile device that needs to be solved, is the pixel coordinate of the scene feature point in the real-time image. The pixel coordinate is the pixel coordinate relative to the real-time image. For map database The pixel coordinates of the matched scene feature points are relative to the pixel coordinates in the database. i ranges from 1 to n, indicating that there are n scene feature points in the real-time image that match the scene feature points in the database. π C is the intrinsic parameter matrix of the camera, which can convert 3D coordinates into pixel coordinates. By (π C ) -1 Convert to get the 3D coordinate relative to the car, and then pass Convert to get 3D coordinates relative to the world coordinate system, and then pass Convert the pixel coordinates relative to the database. is the scene feature point in the database The pose of the key frame relative to the world coordinate system. The pixel coordinates obtained by conversion are the pixel coordinates of the scene feature points that match the database. The reprojection error model is obtained by subtracting the two. Finally, the value of the reprojection error model is minimized by the optimization method to obtain the real-time posture of the car.

[0438] The mobile device performs positioning based on the calculated results. After the network device calculates the positioning results, it returns the positioning results to the mobile device so that the mobile device can perform the positioning operation.

[0439] It should be noted that positioning calculations can also be performed by mobile devices. After the network device determines the same scene feature points in the database, it sends the information of the same scene feature points to the mobile device. The mobile device obtains the posture information according to the preset algorithm and performs the positioning operation.

[0440] In this embodiment, the same scene feature point information sent by the network device to the mobile device specifically includes: pixel coordinates of the scene feature point, key frame pose to which the scene feature point belongs, and may also include 3D coordinates of the scene feature point, which is not limited here.

[0441] In this implementation, the specific process of positioning after the database is built is described. Figure 5 After the database is constructed as shown, since the database contains more descriptor information of different natural conditions, when the database is used for positioning, the real-time image can match more identical descriptor information with the database, thereby making the positioning more accurate.

[0442] It should be noted that, during the positioning process, the database may be updated according to different descriptor information so that the information stored in the database is more complete.

[0443] After determining at least one first descriptor information based on the real-time image, the method further includes comparing the descriptor information preset in the database with the at least one first descriptor information to determine different descriptor information, and constructing the database based on the different descriptor information. Constructing the database based on different descriptor information specifically includes two situations:

[0444] 1. When the second scene feature point information to which different descriptor information belongs does not exist in the database.

[0445] The network device determines whether the second scene feature point exists in its own database. This determination is similar to the determination of whether the scene feature points are identical when the database is constructed, that is, based on the 3D coordinates. The details are not repeated here. If the second scene feature point does not exist in the database, then the second scene feature point information is obviously not present in the database.

[0446] As shown in FIG12( a ), for example, scene feature point 4 does not exist in the database relative to the real-time image, and thus, descriptor 10 for scene feature point 4 does not exist either. Descriptor 10 is a different descriptor in this embodiment. In this case, information about a second scene feature point containing a different descriptor, namely, information about scene feature point 4, is added to the database.

[0447] Second, there is second scene feature point information belonging to different descriptors in the database, but the second scene feature point information in the database does not include target descriptor information.

[0448] When it is determined that the database contains the second scene feature point information, the second scene feature point information does not include the different descriptor information determined above. For example, referring to FIG12(b), the database includes scene feature points 1, 2, and 3 that are identical to those in the real-time image, but the information of scene feature point 3 in the database does not include the target descriptor in scene feature point 3 in the real-time image, i.e., descriptor 9. In this case, the descriptor sub-information corresponding to the different descriptor 9 is added to the database.

[0449] It should be noted that if it is necessary to add a second scene feature point belonging to the target descriptor to the database, the 3D coordinates of the second scene feature point need to be synchronously updated to the database. Since the real-time image information only contains descriptor information and pixel coordinate information, and does not contain 3D coordinates, at this time, the same part of the descriptor in the database is used for positioning first, and then the 3D coordinates of different scene feature points are added to the database. The method for determining the 3D coordinates of different scene feature points is: after using the same descriptor for positioning to obtain the positioning result of the real-time image, the 3D coordinates of different scene feature points are determined by binocular cameras, or by monocular cameras and IMUs. The specific method for determining the 3D coordinates of different scene feature points is not limited here.

[0450] In this embodiment, during the real-time positioning process, the database is continuously improved and updated according to different descriptor information, so that the database can be better utilized for positioning.

[0451] In this embodiment, the real-time positioning process is the process of data interaction between the mobile device and the network device. Figure 13 , which will be explained below:

[0452] 1301. The mobile device sends a real-time image to a network device.

[0453] In addition to sending real-time images to the network device, the mobile device can also send the location information of the mobile device when the real-time image is taken or the natural conditions of the location of the mobile device to the network device.

[0454] 1302. The network device determines at least one first descriptor information according to the real-time image.

[0455] 1303. The network device compares the descriptor information preset in the database with the at least one first descriptor information to determine the same descriptor information and different descriptor information.

[0456] 1304. The network device uses the same descriptor information to locate the real-time image.

[0457] 1305. Send the positioning result to the mobile device.

[0458] The mobile device performs a positioning operation according to the positioning calculation result determined by the network device. In this embodiment, the positioning calculation operation can also be performed by the network device, which is not limited here.

[0459] 1306. The network device constructs a database based on different descriptor information.

[0460] In this embodiment, steps 1301 to 1306 are the same as those in the above embodiment. Figure 11 The steps in the illustrated embodiment are similar and will not be described in detail here.

[0461] The above describes the embodiment of the present application from the perspectives of the positioning method and the database construction method. The structure of the network device in the embodiment of the present application is explained below.

[0462] Based on the above database construction method, a possible structure of the network device is as follows: Figure 14 As shown, including:

[0463] The determining unit 1401 is configured to determine a target image set that meets a preset image coincidence requirement, wherein the target image set includes at least one image, and each image corresponds to a type of natural condition information;

[0464] A processing unit 1402 is configured to obtain a scene feature point information set based on the target image set and the natural condition information corresponding to each image, wherein the scene feature point set includes at least one scene feature point information;

[0465] The determining unit 1401 is further configured to determine, in the scene feature point information set, first scene feature point information corresponding to a scene feature point having a life value greater than a first preset life value threshold for a single mobile device, where the life value indicates a probability that the scene feature point is a static scene feature point;

[0466] The determining unit 1401 is further configured to determine, in the first scene feature point information, second scene feature point information corresponding to a scene feature point where the health value of the plurality of mobile devices is greater than a second preset health value threshold;

[0467] The database construction unit 1403 is configured to construct the database according to the second scene feature point information when the second scene feature point information does not match the scene feature point information preset in the database.

[0468] Optionally, the second scene feature point information does not match scene feature point information preset in the database, including:

[0469] The second scene feature point information does not exist in the database;

[0470] The database construction unit 1403 is specifically configured to add the second scene feature point information to the database, where the second scene feature point information includes target descriptor information about target natural condition information.

[0471] Optionally, the second scene feature point information does not match scene feature point information preset in the database, including:

[0472] The second scene feature point information exists in the database, and the second scene feature point information does not include target descriptor information about target natural condition information;

[0473] The database construction unit 1403 is specifically configured to add the target descriptor information to the second scene feature point information preset in the database.

[0474] Optionally, the determining unit 1401 is further configured to determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0475] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0476] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0477] Optionally, the determining unit 1401 is further configured to determine at least one descriptor information of the second scene feature point preset in a database;

[0478] The network device further includes:

[0479] The judging unit 1404 is configured to judge whether, in the at least one descriptor information, there is a descriptor corresponding to a descriptor information whose distance from the descriptor corresponding to the target descriptor information is less than a preset distance threshold;

[0480] The determining unit 1401 is further configured to determine that the second scene feature point information preset in the database does not include target descriptor information if, in the at least one descriptor information, there is no descriptor corresponding to any descriptor information whose distance to the descriptor corresponding to the target descriptor information is less than a preset distance threshold.

[0481] Optionally, in a fifth achievable manner of the fourth aspect, the life value of the scene feature point on a single mobile device is f, and the calculation formula of f is: Wherein, n represents the number of times the scene feature point is observed in a single mobile device, n0 is the average number of times the preset scene feature point is observed, and σ is the variance of the number of times the preset scene feature point is observed.

[0482] Optionally, the life value of the scene feature point on multiple mobile devices is F, and the calculation formula of F is:

[0483] f is the life value of the scene feature point on a single mobile device, B is a weight coefficient corresponding to each mobile device, and one weight coefficient corresponds to each mobile device among the multiple mobile devices.

[0484] Optionally, the β i The calculation formula is: i =γ t +γ g +γ c , the γ t is the temporal continuity indicator of the scene feature point observed on multiple mobile devices, the γ g is the geometric continuity index of the scene feature points observed on multiple mobile devices, the γ cIt is a description consistency indicator of the scene feature points observed on multiple mobile devices.

[0485] Optionally, the determining unit 1401 is specifically configured to select images according to a preset distance interval;

[0486] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0487] Optionally, the preset distance interval d k+1 ;

[0488] The calculation formula of the preset distance interval is: k+1 =d k +d k (α * -α), wherein the α * is the preset overlap threshold, the d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of overlap between images when selecting images.

[0489] Optionally, the determining unit 1401 is specifically configured to select images according to preset angle intervals;

[0490] When the difference between the degree of coincidence of the selected image and a preset degree of coincidence threshold is within a preset accuracy range, it is determined that the selected image belongs to the target image set.

[0491] Optionally, the preset angle interval θ k+1 ;

[0492] The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ k The angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of overlap between images when selecting images.

[0493] Optionally, the scene feature point information includes descriptor information corresponding to the natural condition information, and the processing unit 1402 is specifically configured to: 1> process one of the target images to obtain scene feature points;

[0494] 2> forming the scene feature point information using the scene feature point, the target image to which the scene feature point belongs, and the natural condition information corresponding to the target image;

[0495] Repeat steps 1> and 2> until the scene feature point information set is obtained.

[0496] Optionally, the determining unit 1401 is further configured to determine the third scene feature point information in the database after the construction is completed;

[0497] The database construction unit 1403 is further configured to delete the third scene feature point information from the database when a feature quantity control score FNCS of the third scene feature point corresponding to the third scene feature point information is less than a preset FNCS threshold.

[0498] Optionally, the calculation formula of the feature quantity control score FNCS is: described is the probability that the scene feature point is used in positioning, It is the ratio of the number of descriptors of the scene feature point to the total number of descriptors in the image to which the scene feature point belongs.

[0499] Based on the above positioning method, another possible structure of the network device is as follows Figure 15 As shown:

[0500] An acquisition unit 1501 is used to acquire real-time images;

[0501] A determining unit 1502 is configured to determine at least one first descriptor information based on the real-time image, wherein the first descriptor information includes information about the natural conditions of the target when the real-time image was captured;

[0502] The determining unit 1502 is further configured to compare descriptor information preset in a database with the at least one first descriptor information to determine identical descriptor information, wherein the network device determines a target image set that satisfies a preset image coincidence requirement for the descriptor information preset in the database, obtains a scene feature point information set based on the target image set and natural condition information corresponding to each image in the target image set, selects first scene feature point information corresponding to a first scene feature point that satisfies a preset life value requirement from the scene feature point information set, and constructs the database based on second descriptor information corresponding to the target natural condition information in the first scene feature point information, wherein the second descriptor information does not match the descriptor information preset in the database, and the scene feature point information includes descriptor information corresponding to the natural condition information;

[0503] The positioning unit 1503 is configured to position the real-time image using the same descriptor information.

[0504] Optionally, the positioning unit 1503 is specifically configured to determine first scene feature point information corresponding to the same descriptor information in a database;

[0505] The position of the target mobile device when the real-time image is captured is calculated based on the first scene feature point information and a positioning calculation formula.

[0506] Optionally, the positioning calculation formula is:

[0507] The location of the target mobile device when the real-time image is captured wherein is the pixel coordinate of the first scene feature point in the real-time image, π C is the intrinsic parameter matrix of the camera, the π C For converting 3D coordinates to pixel coordinates, the is the scene feature point in the database The pose of the image relative to the world coordinate system, is the pixel coordinate of the first scene feature point in the database, the value of i ranges from 1 to n, n is a positive integer, and the first scene feature point corresponds to the first scene feature point information.

[0508] Optionally, the determining unit 1502 is further configured to compare the descriptor information preset in the database with the at least one first descriptor information to determine different descriptor information;

[0509] The network device further includes a database construction unit 1504;

[0510] The database construction unit 1504 is specifically configured to construct the database according to the different descriptor information.

[0511] Optionally, when the second scene feature point information to which the different descriptor information belongs does not exist in the database, the database construction unit 1504 is specifically configured to add the second scene feature point information containing the different descriptor information to the database.

[0512] Optionally, when the second scene feature point information to which the different descriptor information belongs exists in the database, the database construction unit 1504 is specifically configured to add the different descriptor information to the second scene feature point information in the database.

[0513] Optionally, the determining unit 1502 is further configured to determine the 3D coordinates of the second scene feature point corresponding to the second scene feature point information;

[0514] When the difference between the 3D coordinates of each scene feature point preset in the database and the 3D coordinates of the second scene feature point is greater than a first preset threshold, determining that the second scene feature point information does not exist in the database;

[0515] When the difference between the 3D coordinates of any scene feature point preset in the database and the 3D coordinates of the second scene feature point is less than the first preset threshold, it is determined that the second scene feature point information exists in the database.

[0516] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and no further details will be given here.

[0517] An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a program, and the program executes some or all of the steps recorded in the above method embodiment.

[0518] Next, another network device provided by the embodiment of the present application is introduced. Figure 16 As shown, the network device 1600 includes:

[0519] Receiver 1601, transmitter 1602, processor 1603 and memory 1604 (wherein the number of processor 1603 in network device 1600 can be one or more, Figure 16 In some embodiments of the present application, the receiver 1601, the transmitter 1602, the processor 1603 and the memory 1604 may be connected via a bus or other means, wherein: Figure 16 The bus connection is taken as an example.

[0520] Memory 1604 may include read-only memory and random access memory, and provides instructions and data to processor 1603. A portion of memory 1604 may also include non-volatile random access memory (NVRAM). Memory 1604 stores an operating system and operating instructions, executable modules, or data structures, or subsets or extensions thereof. The operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and processing hardware-based tasks.

[0521] Processor 1603 controls the operation of the network device and may also be referred to as a central processing unit (CPU). In specific applications, the various components of the network device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0522] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1603. Processor 1603 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be performed by hardware integrated logic circuits or software instructions in processor 1603. The above processor 1603 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1604, and processor 1603 reads the information in memory 1604 and, in conjunction with its hardware, completes the steps of the above method.

[0523] The receiver 1601 may be used to receive input digital or character information, and to generate signal input related to the relevant settings and function control of the network device. The transmitter 1602 may include a display device such as a display screen, and the transmitter 1602 may be used to output digital or character information through an external interface.

[0524] In the embodiment of the present application, the processor 1603 is used to execute the aforementioned database construction method and positioning method.

[0525] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0526] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or dedicated circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0527] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0528] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive Solid State Disk (SSD)), etc.

Claims

1. A positioning method, characterized in that: Applied to vehicles, including: Acquiring real-time images through a vehicle-mounted camera, and information about natural conditions when acquiring the real-time images; Sending the real-time image and the natural condition information to a server, and receiving positioning information fed back by the server, the positioning information being obtained by the server based on the real-time image and the natural condition information, the positioning information including coordinate information of a plurality of scene feature points, wherein the plurality of scene feature points are stored in the server and matched with the real-time image and the natural condition information, the description information of at least one scene feature point among the plurality of scene feature points being the same as the natural condition information, the natural condition information including: viewing direction, weather and / or lighting conditions; Obtaining the position information of the vehicle based on the positioning information includes: determining the position information of the vehicle based on the coordinate information of the multiple scene feature points and the pixel coordinates of the multiple scene feature points in the real-time image, wherein the coordinate information of the multiple scene feature points includes world coordinates and pixel coordinates.

2. The method according to claim 1, characterized in that The positioning information is the location information of the vehicle.

3. The method according to claim 1, characterized in that The method further comprises: Acquiring images based on a preset distance interval or a preset angle interval; When the difference between the obtained degree of coincidence of the images and a preset degree of coincidence threshold is within a preset accuracy range, determining that the selected image belongs to the target image set; The target image set is sent to a server.

4. The method according to claim 3, characterized in that The method further comprises: Calculate the preset distance interval d k+1 ; The calculation formula of the preset distance interval is: k+1 =d k +d k (α * -α), wherein the α * is the preset overlap threshold, the d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of overlap between images when selecting images.

5. The method according to claim 3, characterized in that The method further comprises: Calculate the preset angle interval θ k+1 ; The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ k The angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of overlap between images when selecting images.

6. A vehicle, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is configured to execute the program in the memory, comprising the following steps: Acquiring real-time images through a vehicle-mounted camera, and information about natural conditions when acquiring the real-time images; Sending the real-time image and the natural condition information to a server, and receiving positioning information fed back by the server, the positioning information being obtained by the server based on the real-time image and the natural condition information, the positioning information including coordinate information of a plurality of scene feature points, wherein the plurality of scene feature points are stored in the server and matched with the real-time image and the natural condition information, the description information of at least one scene feature point among the plurality of scene feature points being the same as the natural condition information, the natural condition information including: viewing direction, weather and / or lighting conditions; Obtaining the position information of the vehicle based on the positioning information includes: determining the position information of the vehicle based on the coordinate information of the multiple scene feature points and the pixel coordinates of the multiple scene feature points in the real-time image, wherein the coordinate information of the multiple scene feature points includes world coordinates and pixel coordinates.

7. The vehicle according to claim 6, characterized in that The positioning information is the location information of the vehicle.

8. The vehicle according to claim 6, characterized in that The processor is further configured to: Acquiring images based on a preset distance interval or a preset angle interval; When the difference between the obtained degree of coincidence of the images and a preset degree of coincidence threshold is within a preset accuracy range, determining that the selected image belongs to the target image set; The target image set is sent to a server.

9. The vehicle according to claim 8, characterized in that The processor is further configured to: Calculate the preset distance interval d k+1 ; The calculation formula of the preset distance interval is: k+1 =d k +d k (α * -α), wherein the α * is the preset overlap threshold, the d k The distance interval of the image selected at the previous moment, α is the distance interval d k The degree of overlap between images when selecting images.

10. The vehicle according to claim 8, characterized in that The processor is further configured to: Calculate the preset angle interval θ k+1 ; The calculation formula of the preset angle interval is: θ k+1 =θ k +θ k (α * -α), wherein the α * is the preset overlap threshold, the θ k The angle interval of the image selected at the previous moment, α is the angle interval θ k The degree of overlap between images when selecting images.

11. A computer-readable storage medium comprising instructions, which, when executed on a computer, enable the computer to perform the method according to any one of claims 1 to 5.

Images