Visual positioning method, visual positioning and mapping method, device, equipment and medium

Abstract

Embodiments of the present disclosure relate to a visual positioning method, a visual positioning and mapping method, an apparatus, a device, and a medium. The visual positioning method includes: obtaining positioning information of a vehicle and corresponding initial weights, and collecting an image. The positioning information includes a first vehicle positioning determined by a visual positioning method and a second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight; obtaining an image quality score of the collected image, where the image quality score represents the proportion of the invalid area; reallocating the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight; and performing positioning fusion based on the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain a fused positioning result of the vehicle. Embodiments of the present disclosure greatly reduce the positioning error caused by low image quality, thereby improving the accuracy and robustness of visual positioning and mapping.

Description

Technical Field

[0001] The present disclosure relates to the field of positioning technology, and in particular, to a visual positioning method, a visual positioning and mapping method, a device, a device and a medium. Background Art

[0002] A positioning system is one of the most important components in a vehicle, especially an intelligent driving vehicle. The positioning technology adopted by the positioning system can include various types. The visual simultaneous localization and mapping (VSLAM) technology, due to its advantages of high precision and low cost, can be used as a key means to solve the vehicle positioning problem.

[0003] Visual SLAM usually relies on rich texture information and has high requirements for image quality. In the case of few visual features and low quality in the image, this positioning method is prone to failure, resulting in large positioning errors. At the same time, redundant low-quality features will also be unexpectedly added to the map. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a visual positioning method, a visual positioning and mapping method, a device, a device and a medium.

[0005] An embodiment of the present disclosure provides a visual positioning method, the method comprising:

[0006] Obtaining the positioning information of the vehicle and the corresponding initial weights, and collecting images, where the positioning information includes the first vehicle positioning determined by a visual positioning method and the second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight;

[0007] Obtaining the image quality score of the collected images, where the image quality score represents the proportion of the invalid area;

[0008] Reallocating the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight;

[0009] Performing positioning fusion according to the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fusion positioning result of the vehicle.

[0010] An embodiment of the present disclosure also provides a visual positioning and mapping method, the method comprising:

[0011] Obtaining the collected images of the vehicle;

[0012] Obtain the image quality score of the acquired image, where the image quality score characterizes the proportion of the invalid area;

[0013] Extract multiple real-time visual features and multiple existing map visual features from the acquired image, and match the multiple real-time visual features among the multiple map visual features to determine the second real-time visual feature with a matching failure;

[0014] Set the confidence of the second real-time visual feature according to the image quality score, where the higher the image quality score, the lower the confidence;

[0015] Based on the fusion positioning result of the vehicle, the second real-time visual feature and its confidence, input them into the visual mapping module to create a visual map.

[0016] The embodiment of the present disclosure also provides a visual positioning device, and the device includes:

[0017] An acquisition module, configured to acquire the positioning information of the vehicle, the corresponding initial weight, and the acquired image, where the positioning information includes the first vehicle positioning determined by the visual positioning method and the second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight;

[0018] An image quality module, configured to obtain the image quality score of the acquired image, where the image quality score characterizes the proportion of the invalid area;

[0019] A weight distribution module, configured to re-distribute the initial weight of the first vehicle positioning according to the image quality score to obtain a re-distributed weight;

[0020] A positioning module, configured to perform positioning fusion according to the re-distributed weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fusion positioning result of the vehicle.

[0021] The embodiment of the present disclosure also provides a visual positioning and mapping device, and the device includes:

[0022] An image module, configured to acquire the acquired image of the vehicle;

[0023] A quality module, configured to obtain the image quality score of the acquired image, where the image quality score characterizes the proportion of the invalid area;

[0024] A matching failure module, configured to extract multiple real-time visual features and multiple existing map visual features from the acquired image, and match the multiple real-time visual features among the multiple map visual features to determine the second real-time visual feature with a matching failure;

[0025] A confidence module, configured to set the confidence of the second real-time visual feature according to the image quality score, where the higher the image quality score, the lower the confidence;

[0026] A map module, configured to input the fused positioning result of the vehicle, the second real-time visual feature, and its confidence into a visual mapping module to create a visual map.

[0027] An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing executable instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the visual positioning method or the visual positioning and mapping method provided by the embodiment of the present disclosure.

[0028] An embodiment of the present disclosure further provides a computer-readable storage medium, storing a computer program for executing the visual positioning method or the visual positioning and mapping method provided by the embodiment of the present disclosure.

[0029] The technical solution provided by the embodiment of the present disclosure has the following advantages compared with the prior art: The visual positioning solution provided by the embodiment of the present disclosure obtains the positioning information of the vehicle and the corresponding initial weights, and acquires images. The positioning information includes the first vehicle positioning determined by the visual positioning method and the second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight; obtains the image quality score of the acquired image, and the image quality score represents the proportion of the invalid area; reallocates the initial weight of the first vehicle positioning according to the image quality score to obtain the reallocated weight; performs positioning fusion according to the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fused positioning result of the vehicle. By adopting the above technical solution, through real-time quality evaluation of the acquired images, the weights of the vehicle positioning in the visual positioning method are reallocated according to the quality of the acquired images, and then the final positioning result is obtained through positioning fusion with the adjusted weights and positions. The positioning weights are strongly correlated with the image quality, greatly reducing the positioning error caused by low image quality, and thus improving the accuracy and robustness of visual positioning. Description of the Drawings

[0030] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0031] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0032] Figure 1 A flowchart of a visual positioning method provided by an embodiment of the present disclosure;

[0033] Figure 2 A training diagram of an image quality determination model provided by an embodiment of the present disclosure;

[0034] Figure 3 A flowchart of a visual positioning and mapping method provided by an embodiment of the present disclosure;

[0035] Figure 4 A schematic diagram of a visual positioning and mapping system provided by an embodiment of the present disclosure;

[0036] Figure 5 A schematic diagram of the structure of a visual positioning device provided by an embodiment of the present disclosure;

[0037] Figure 6 A schematic diagram of the structure of a visual positioning and mapping device provided by an embodiment of the present disclosure;

[0038] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0039] In order to be able to more clearly understand the above objects, features and advantages of the present disclosure, the following will further describe the solutions of the present disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0040] In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all the embodiments.

[0041] Visual SLAM usually relies on rich texture information and has high requirements for image quality. In the case of unsatisfactory lighting conditions (such as too strong or too dark) and a large proportion of low-texture areas (such as walls) in the image, since there are few visual features and low quality in the image, this positioning method is prone to failure, resulting in large positioning errors. At the same time, redundant low-quality features will also be unexpectedly added to the map. To solve the above problems, the embodiments of the present disclosure provide a visual positioning method and a visual positioning and mapping method, which will be introduced below in combination with specific embodiments.

[0042] Figure 1 FIG. is a schematic flowchart of a visual positioning method provided by an embodiment of the present disclosure. This method can be executed by a visual positioning device, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 1 shown, the method includes:

[0043] Step 101, obtain the positioning information of the vehicle and the corresponding initial weights, and collect images. The positioning information includes the first vehicle positioning determined by the visual positioning method and the second vehicle positioning determined by at least one other positioning method. Each vehicle positioning has a corresponding initial weight.

[0044] The vehicle targeted by the embodiments of the present disclosure can be various types of vehicles, such as driverless vehicles or intelligent driving vehicles, etc., without specific limitation. The positioning information can be the current position information of the vehicle, which can specifically include the vehicle positioning determined by various positioning methods. In the embodiments of the present disclosure, the positioning information can include the first vehicle positioning determined by the visual positioning method and the second vehicle positioning determined by at least one other positioning method. The visual positioning method can be understood as vehicle positioning based on visual features. The other positioning methods are not specifically limited. For example, they can include positioning by GPS, Inertial Measurement Unit (IMU), or by vehicle odometer, etc. When there are multiple other positioning methods, the number of the second vehicle positionings is multiple. The initial weight can be a weight determined inside the positioning module of different positioning methods.

[0045] Specifically, the visual positioning device can obtain the positioning information of the vehicle and the corresponding initial weights from the visual positioning module and other positioning modules, and obtain the collected images collected by the image acquisition device in real time for subsequent use.

[0046] Step 102, obtain the image quality score of the collected image. The image quality score represents the proportion of the invalid area.

[0047] Among them, the image quality score can be a parameter output by an image quality determination model for characterizing the image quality. Specifically, the image quality is represented by the proportion of the invalid area. The invalid area can be understood as an area where the pixel values change little and is visually unhelpful. In the embodiments of the present disclosure, the invalid area includes at least one of an overexposed area, an underexposed area, and a low-texture area.

[0048] In some embodiments, obtaining the image quality score of the captured image may include: inputting the captured image into a pre-trained image quality determination model to obtain the image quality score. The image quality determination model can be a newly added deep learning model in the embodiments of the present disclosure for real-time evaluation of the quality of the captured image.

[0049] In the embodiments of the present disclosure, after the visual positioning device obtains the captured image, it can input the captured image into a pre-trained image quality determination model. After model calculation, the image quality score is output. The larger the image quality score, the higher the proportion of the invalid area, that is, the lower the image quality.

[0050] Step 103: Reassign the initial weight of the first vehicle positioning according to the image quality score to obtain a reassigned weight.

[0051] In some embodiments, reassigning the initial weight of the first vehicle positioning according to the image quality score to obtain a reassigned weight includes: increasing or decreasing the initial weight of the first vehicle positioning according to the size of the image quality score to obtain a reassigned weight, where the reassigned weight of the first vehicle positioning is inversely proportional to the image quality score.

[0052] After the visual positioning device determines the image quality score, it can reassign the initial weight of the first vehicle positioning according to the image quality score, that is, adjust the initial weight of the first vehicle positioning, so that the adjusted reassigned weight is smaller when the image quality score is larger and larger when the image quality score is smaller. The specific adjustment method can include increasing or decreasing. For example, reducing the initial weight with a larger image quality score by a larger value, and setting a smaller reduction value for the initial weight with a smaller image quality score; or increasing the initial weight with a smaller image quality score by a larger value, and setting a smaller increase value for the initial weight with a smaller image quality score, both can achieve the above results.

[0053] Optionally, the reassigned weight can be equal to the product of the initial weight and the target score, where the target score is the difference between 1 and the image quality score, and the value of the image quality score ranges from 0 to 1.

[0054] The reallocated weight can be determined by the formula w′ = w(1 - s), where w represents the initial weight of the first vehicle positioning, s represents the image quality score, w′ is the reallocated weight of the visual positioning, and (1 - s) represents the above-mentioned target score. The value of the image quality score ranges from 0 to 1, and the value of the target score also ranges from 0 to 1. For example, when the initial weight is 0.5 and the image quality score is 0.8, the reallocated weight is 0.5(1 - 0.8) = 0.1.

[0055] Step 104: Perform positioning fusion based on the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fused positioning result of the vehicle.

[0056] After the visual positioning device reallocates the initial weight of the first vehicle positioning according to the image quality score to obtain the reallocated weight, it can perform positioning fusion processing on the first vehicle positioning and the second vehicle positioning according to the corresponding weights to obtain the final fused positioning result of the vehicle. Specifically, it can be calculated through the formula where T′ represents the fused positioning result of the vehicle, and T i represents the i-th vehicle positioning, including the above-mentioned first vehicle positioning and at least one second vehicle positioning, i = 1, 2,..., n, and n represents the total number of the above-mentioned first vehicle positioning and at least one second vehicle positioning, and w i represents the weight corresponding to the i-th vehicle positioning. For example, when i = 1, it represents the reallocated weight of the first vehicle positioning, and when i = 2, it represents the initial weight of a second vehicle positioning.

[0057] In this solution, after reallocating the weights of the vehicle positioning determined by the visual positioning method, the weights of low-quality acquired images can be reduced. The positioning weight is strongly correlated with the image quality, which greatly reduces the positioning error caused by low image quality and improves the accuracy of vehicle positioning.

[0058] The visual positioning solution provided by the embodiments of the present disclosure obtains the positioning information of the vehicle and the corresponding initial weights, and acquires images. The positioning information includes the first vehicle positioning determined by visual positioning and the second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight; obtains the image quality score of the acquired image, and the image quality score represents the proportion of the invalid area; reallocates the initial weight of the first vehicle positioning according to the image quality score to obtain the reallocated weight; performs positioning fusion according to the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fused positioning result of the vehicle. By adopting the above technical solution, through real-time quality evaluation of the acquired images, the weights of the vehicle positioning in the visual positioning method are reallocated according to the quality of the acquired images, and then positioning fusion is performed through the adjusted weights and positions to obtain the final positioning result. The positioning weights are strongly correlated with the image quality, greatly reducing the positioning error caused by low image quality, and thus improving the accuracy and robustness of visual positioning.

[0059] In some embodiments, the image quality determination model is obtained by training an initial model based on a neural network with respect to sample images and the proportion annotations of the corresponding invalid areas of the sample images.

[0060] In the embodiments of the present disclosure, when training the image quality determination model, a relatively large number of sample images can be obtained first, and the proportion of the invalid area of each sample image is annotated, and the annotation value is between 0 and 1, and the annotation is performed according to the proportion of the pixels of the invalid area relative to the entire sample image. Then, the sample images are used as inputs, and the proportion annotations of the corresponding invalid areas of the sample images are used as outputs to train the initial model based on a neural network, and the initial model with the finally trained parameters is determined as the image quality determination model.

[0061] In some embodiments, the image quality determination model includes a feature extraction module, an attention module, and an output module. The feature extraction module is used to extract image features, the attention module is used to enhance the weights of the invalid areas of the image, and the output module is used to output the image quality score determined according to the proportion of the invalid area.

[0062] Exemplarily, Figure 2 is a training schematic diagram of an image quality determination model provided by the embodiments of the present disclosure, as Figure 2As shown in the figure, the backbone module in the figure is the feature extraction module, which is used to extract features from the collected images, and can use but not limited to ResetNet, MobileNet, etc.; the attention module can include a max pooling layer (Maxpool), a convolution layer (Convolution), and a sigmoid function (Sigmoid) network layer, which are used to enhance the weight of the invalid area of the collected image, and output a weight map with a value between 0 and 1. This weight map will be multiplied by the output features of the backbone network to obtain the final features; the output module includes an average pooling layer (AvgPool) and a sigmoid function (Sigmoid) network layer, which are used to output the proportion of the invalid area between 0 and 1, that is, the image quality score. The error between the output score and the supervision of the annotation module is used to update the network parameters in a backpropagation manner to realize the training of the model; after the training is completed, the image quality determination model is used to perform real-time image quality evaluation to obtain the image quality score of the collected image.

[0063] In the above solution, the proportion of the invalid area in the image is used to represent the image quality, and a relatively accurate image quality determination model is obtained through the training of the deep learning model. It can not only quantitatively evaluate the image quality, but also has a high evaluation accuracy, which is beneficial to the subsequent allocation of visual positioning weights based on the image quality.

[0064] In some embodiments, the first vehicle positioning determined by the visual positioning method may include: extracting a plurality of real-time visual features and a plurality of existing map visual features from the collected image, and the map visual features have corresponding confidence levels; matching the plurality of real-time visual features among the plurality of map visual features to determine the first real-time visual feature that matches successfully; based on the first real-time visual feature, the first map visual feature that matches the first real-time visual feature, the confidence level of the first map visual feature, and the error calculation function, determining the vehicle positioning with the smallest error as the first vehicle positioning.

[0065] Among them, the real-time visual feature can be understood as the visual feature obtained by real-time monitoring of the collected image, including but not limited to low-level visual features and high-level visual features. The low-level visual features include ORB (Oriented Fast and Rotated Brief features, etc., and the high-level visual features include lane lines, etc. The map visual feature can be the visual feature included in the visual map that has been created in the visual mapping module. Each map visual feature has a confidence level assigned before inputting into the visual mapping module. In the embodiments of the present disclosure, this confidence level can be strongly correlated with the above image quality score, and the larger the image quality score, the lower the confidence level.

[0066] In an embodiment of the present disclosure, after obtaining a captured image, a visual positioning device may extract multiple real-time visual features from the captured image, and obtain multiple existing map visual features and their corresponding confidence levels from a visual mapping module; then each real-time visual feature may be matched among the multiple map visual features, and the specific matching method is not limited. For example, the matching result may be determined based on feature similarity; determine the first real-time visual features among the multiple real-time visual features whose matching results are successful, and the number of the first real-time visual features may also be multiple; then based on the first real-time visual features, the first map visual features matched with the first real-time visual features, the confidence levels of the first map visual features, and an error calculation function, determine that the vehicle positioning when the error is minimized is the first vehicle positioning.

[0067] Optionally, the formula representation of the visual positioning method is:

[0068] T* = argmin∑c*e(T, f, f_map),

[0069] where argmin represents the optimal solution of vehicle positioning that makes ∑c*e(T, f, f_map) obtain the minimum value, that is, T* represents the first vehicle positioning, e represents the error calculation function, and the error includes but is not limited to reprojection error, etc. T represents vehicle positioning and is the independent variable of the function on the right side of the equal sign, f represents the first real-time visual feature, f_map represents the first map visual feature matched with the first real-time visual feature, and c represents the confidence level of the first map visual feature.

[0070] Input each of the above first real-time visual features, the first map visual features matched with the first real-time visual features, and the confidence levels of the first map visual features into the above formula, and calculate the variable value that makes ∑c*e(T, f, f_map) obtain the minimum value, that is, calculate that the vehicle positioning when the error is minimized is the above first vehicle positioning. The formula of the above visual positioning method is only an example, not a limitation.

[0071] In the above solution, when determining the corresponding vehicle positioning through the visual positioning method, it may be determined based on the matching result between the real-time visual feature and the existing map visual feature and the error calculation function, and then the vehicle positioning can be fused with the vehicle positioning in other ways based on this vehicle positioning to obtain a more accurate fused positioning result of the vehicle.

[0072] Figure 3 It is a schematic flowchart of a visual positioning and mapping method provided by an embodiment of the present disclosure. This method may be executed by a visual positioning and mapping device, where the device may be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 3 shown, this method includes:

[0073] Step 301, obtain a captured image of the vehicle.

[0074] Specifically, the visual positioning and mapping device can obtain the captured images captured in real time by the image capturing device for subsequent use.

[0075] Step 302: Obtain the image quality score of the captured image, where the image quality score represents the proportion of the invalid area.

[0076] Among them, the image quality score can be a parameter output by the image quality determination model for characterizing the image quality. Specifically, the image quality is represented by the proportion of the invalid area. The invalid area can be understood as an area with relatively small pixel value changes, which is not helpful for vision. In the embodiments of the present disclosure, the invalid area includes at least one of an overexposed area, an underexposed area, and a low-texture area.

[0077] In the embodiments of the present disclosure, the image quality score is obtained by inputting the captured image into a pre-trained image quality determination model. For the specific determination process of the image quality score, refer to the above embodiments and will not be elaborated here.

[0078] Step 303: Extract a plurality of real-time visual features and a plurality of existing map visual features from the captured image, and match the plurality of real-time visual features among the plurality of map visual features to determine the second real-time visual features that fail to match.

[0079] Among them, the real-time visual features can be understood as the visual features obtained by real-time monitoring of the captured image. The map visual features can be the visual features included in the visual map that has been created in the visual mapping module. Each map visual feature has a confidence level assigned before being input into the visual mapping module. In the embodiments of the present disclosure, this confidence level can be strongly correlated with the above image quality score, and the higher the image quality score, the lower the confidence level.

[0080] The visual positioning and mapping device can match each real-time visual feature among the plurality of map visual features. The specific matching method is not limited. For example, the matching result can be determined based on feature similarity; determine the second real-time visual features that fail to match among the plurality of real-time visual features, and the number of the second real-time visual features can also be multiple.

[0081] Step 304: Set the confidence level of the second real-time visual features according to the image quality score, where the higher the image quality score, the lower the confidence level.

[0082] Among them, the confidence level can be understood as a parameter representing the degree of credibility or reliability.

[0083] The visual positioning and mapping device can set the confidence level of the second real-time visual features that fail to match according to the above image quality score. The higher the image quality score, the lower the confidence level, that is, the lower the image quality, the lower the confidence level.

[0084] Optionally, the confidence level is equal to the difference between 1 and the image quality score, and the value of the confidence level ranges from 0 to 1.

[0085] When setting the confidence level of the second real-time visual feature according to the image quality score, the following formula c = 1 - s can be used, where s represents the image quality score, c represents the confidence level of the second real-time visual feature, and the value of the confidence level ranges from 0 to 1. The above is only an example, and the confidence level can also be determined by other formulas, as long as it satisfies that the higher the image quality score, the lower the confidence level.

[0086] Step 305: Based on the fusion positioning result of the vehicle, the second real-time visual feature, and its confidence level, input them into the visual mapping module to create a visual map.

[0087] After setting the confidence level of the second real-time visual feature, the visual positioning and mapping device can input the fusion positioning result of the vehicle, the second real-time visual feature, and its set confidence level into the visual mapping module to create a visual map, and obtain the corresponding map visual feature, so as to determine the corresponding vehicle positioning in the next moment by using the visual positioning method.

[0088] Among them, the fusion positioning result of the vehicle is obtained by performing positioning fusion on the third vehicle positioning determined by using the visual positioning method and the fourth vehicle positioning determined by at least one other positioning method; determining the third vehicle positioning by using the visual positioning method may include: obtaining the third real-time visual feature that matches successfully; based on the third real-time visual feature, determining the third vehicle positioning by using the visual positioning method.

[0089] Optionally, based on the third real-time visual feature, determining the third vehicle positioning by using the visual positioning method includes: based on the third real-time visual feature, the second map visual feature that matches the third real-time visual feature, the confidence level of the second map visual feature, and the error calculation function, determining that the vehicle positioning when the error is the smallest is the third vehicle positioning, where the confidence level of the second map visual feature is set based on the historical image quality score.

[0090] The determination of the fusion positioning result of the vehicle is the same as the method in the above embodiment, and determining the third vehicle positioning by using the visual positioning method is the same as the method of determining the first vehicle positioning by using the visual positioning method in the above embodiment. The third real-time visual feature, the second map visual feature that matches the third real-time visual feature, and the confidence level of the second map visual feature can also be input into the formula of the above visual positioning method, and the vehicle positioning when the error is the smallest is calculated as the third vehicle positioning. For the specific process, refer to the above embodiment and will not be elaborated here.

[0091] In the above solution, for the real-time visual features that successfully match the map visual features, the corresponding vehicle positioning can be determined through visual positioning; for the real-time visual features that fail to match the map visual features, confidence levels can be assigned, which is beneficial for subsequent visual mapping.

[0092] In some embodiments, the visual positioning and mapping method may further include: after the visual map is created, determining the fused positioning result of the vehicle based on the created visual map. At this time, the visual map is not updated after the fused positioning result of the vehicle is determined.

[0093] The visual positioning solution provided by the embodiments of the present disclosure obtains the acquisition image of the vehicle; obtains the image quality score of the acquisition image, where the image quality score represents the proportion of the invalid area; extracts a plurality of real-time visual features and a plurality of existing map visual features from the acquisition image, and matches the plurality of real-time visual features among the plurality of map visual features to determine the second real-time visual features that fail to match; sets the confidence level of the second real-time visual features according to the image quality score, where the higher the image quality score, the lower the confidence level; inputs the fused positioning result of the vehicle, the second real-time visual features, and their confidence levels into the visual mapping module to create a visual map. By adopting the above technical solution, by matching the real-time visual features of the acquisition image with the existing map visual features and performing real-time quality assessment on the acquisition image, setting the confidence level of the real-time visual features that fail to match according to the quality of the acquisition image, and then creating a map based on the real-time visual features, their confidence levels, and the fused positioning result of the vehicle. Since the confidence level is strongly correlated with the image quality, redundant low-quality features are avoided from being unexpectedly added to the map, thereby avoiding the negative impact of low-quality features in the map and improving the accuracy and robustness of visual positioning and mapping.

[0094] Next, a specific example is used to further illustrate the above visual positioning and mapping process. Exemplarily, Figure 4 is a schematic diagram of a visual positioning and mapping system provided by the embodiments of the present disclosure, as Figure 4As shown in the figure, the visual positioning and mapping system may include a positioning module and a visual mapping module in the figure. The positioning module may include a visual positioning module, other positioning source modules, an image quality evaluation module, and a fusion module. The visual positioning module may obtain existing map visual features from the visual mapping module, extract real-time visual features from the acquired images, match the real-time visual features in the map visual features, determine the successfully matched real-time visual features and the failed-to-match real-time visual features, determine the above-mentioned first vehicle positioning or third vehicle positioning based on the successfully matched real-time visual features, and then may output the first vehicle positioning or third vehicle positioning, the initial weight, and the failed-to-match real-time visual features; then in the fusion module, the initial weight for the first vehicle positioning or third vehicle positioning is reallocated based on the image quality score of the acquired image obtained by the image quality evaluation module, and then may perform positioning fusion with the positioning and weight of other positioning source modules to obtain the fused positioning result of the final vehicle and input it into the visual mapping module, and also input the confidence assignment for the failed-to-match real-time visual features into the visual mapping module after the confidence assignment; after receiving the fused positioning result of the vehicle, the failed-to-match real-time visual features, and their confidence, the visual mapping module may perform mapping until the visual map is created, and then may continue to determine the fused positioning result of the vehicle based on the created visual map.

[0095] The visual positioning solution and the visual positioning and mapping solution provided by this solution address the defect of weak processing ability for low-quality images in the related art. Based on the deep learning algorithm, the acquired images are evaluated for real-time quality. By reducing the positioning assignment weight of low-quality images and the confidence of their corresponding visual features, the risk of generating large positioning errors and the negative impact of low-quality features in the map are reduced, thereby improving the accuracy and robustness of the visual positioning and mapping system and enhancing the overall performance of intelligent driving vehicles. It can be applied to various scenarios using visual mapping and positioning systems, such as parking.

[0096] Figure 5 It is a schematic structural diagram of a visual positioning device provided by an embodiment of the present disclosure; this device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 5 shown, this device includes:

[0097] An acquisition module 501, configured to acquire the positioning information of the vehicle, the corresponding initial weight, and the acquired image. The positioning information includes the first vehicle positioning determined by the visual positioning method and the second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight;

[0098] An image quality module 502, configured to acquire the image quality score of the acquired image, and the image quality score represents the proportion of the invalid area;

[0099] A weight distribution module 503, configured to re - distribute the initial weight of the first vehicle positioning according to the image quality score to obtain a re - distributed weight;

[0100] A positioning module 504, configured to perform positioning fusion according to the re - distributed weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain a fused positioning result of the vehicle.

[0101] Optionally, the image quality module 502 is configured to:

[0102] Input the acquired image into a pre - trained image quality determination model to obtain an image quality score, where the image quality determination model is trained based on a neural network initial model with respect to sample images and the proportion annotation of the corresponding invalid regions of the sample images.

[0103] Optionally, the image quality determination model includes a feature extraction module, an attention module, and an output module. The feature extraction module is configured to extract image features, the attention module is configured to enhance the weight of the invalid regions of the image, and the output module is configured to output an image quality score determined according to the proportion of the invalid regions.

[0104] Optionally, the invalid region represents a region with relatively small pixel value changes, and the invalid region includes at least one of an over - exposed region, an over - dark region, and a low - texture region.

[0105] Optionally, the weight distribution module 503 is configured to:

[0106] Increase or decrease the initial weight of the first vehicle positioning according to the magnitude of the image quality score to obtain a re - distributed weight, where the re - distributed weight of the first vehicle positioning is inversely proportional to the image quality score, and the larger the image quality score, the higher the proportion of the invalid region is characterized.

[0107] Optionally, the re - distributed weight is equal to the product of the initial weight and a target score, where the target score is the difference between 1 and the image quality score, and the value of the image quality score ranges from 0 to 1.

[0108] Optionally, the device further includes a first positioning module, configured to:

[0109] Extract a plurality of real - time visual features and a plurality of existing map visual features from the acquired image, where the map visual features have corresponding confidence levels;

[0110] Match the plurality of real - time visual features among the plurality of map visual features to determine a first real - time visual feature with successful matching;

[0111] Based on the first real-time visual feature, the first map visual feature that matches the first real-time visual feature, the confidence level of the first map visual feature, and an error calculation function, determine that the vehicle positioning when the error is minimized is the first vehicle positioning.

[0112] The visual positioning device provided by the embodiments of the present disclosure can execute the visual positioning method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0113] Figure 6 The following is a schematic structural diagram of a visual positioning and mapping device provided by an embodiment of the present disclosure. This device can be implemented by software and / or hardware, and is generally integrated in an electronic device. As Figure 6 shown, the device includes:

[0114] An image module 601, configured to acquire a captured image of a vehicle;

[0115] A quality module 602, configured to acquire an image quality score of the captured image, where the image quality score represents the proportion of an invalid area;

[0116] A matching failure module 603, configured to extract multiple real-time visual features from the captured image and multiple existing map visual features, and match the multiple real-time visual features among the multiple map visual features to determine a second real-time visual feature with a matching failure;

[0117] A confidence level module 604, configured to set a confidence level of the second real-time visual feature according to the image quality score, where the higher the image quality score, the lower the confidence level;

[0118] A map module 605, configured to input the fusion positioning result of the vehicle, the second real-time visual feature, and its confidence level into a visual mapping module to create a visual map.

[0119] Optionally, the confidence level is equal to the difference between 1 and the image quality score, and the value of the confidence level is between 0 and 1.

[0120] Optionally, the image quality score is obtained by inputting the captured image into a pre-trained image quality determination model.

[0121] Optionally, the fusion positioning result of the vehicle is obtained by performing positioning fusion on a third vehicle positioning determined by using a visual positioning method and a fourth vehicle positioning determined by at least one other positioning method;

[0122] The device further includes a third positioning module, configured to:

[0123] Obtain the third real-time visual feature that matches successfully;

[0124] Based on the third real-time visual feature, determine the third vehicle positioning by using visual positioning.

[0125] Optionally, the third positioning module is used for:

[0126] Based on the third real-time visual feature, the second map visual feature that matches the third real-time visual feature, the confidence of the second map visual feature, and the error calculation function, determine the vehicle positioning with the smallest error as the third vehicle positioning, where the confidence of the second map visual feature is set based on the historical image quality score.

[0127] Optionally, the device further includes a continued positioning module, which is used for:

[0128] After the visual map is created, determine the fusion positioning result of the vehicle based on the created visual map.

[0129] The visual positioning and mapping device provided by the embodiments of the present disclosure can execute the visual positioning and mapping method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0130] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As Figure 7 shown, the electronic device 700 includes a central processing unit (CPU) 701, which can execute various processes in the foregoing embodiments according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 into the random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0131] The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. The driver 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the driver 710 as needed, so that the computer program read from it can be installed into the storage part 708 as needed.

[0132] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly embodied on a machine-readable medium. The computer program includes program code for performing the foregoing visual positioning method and / or visual positioning and mapping method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and / or installed from the removable medium 711.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of apparatuses, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0134] The units or modules involved in the embodiments described in the present disclosure can be implemented in software or in hardware. The units or modules described can also be provided in a processor, and the names of these units or modules do not, in some cases, constitute a limitation on the units or modules themselves.

[0135] In addition, the present disclosure also provides a computer-readable storage medium, which can be the computer-readable storage medium included in the device described in the above embodiment; or it can exist separately and be a computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the visual positioning method and / or visual positioning and mapping method described in the present disclosure.

[0136] In addition to the above methods and devices, an embodiment of the present disclosure can also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the visual positioning method and / or visual positioning and mapping method provided in the embodiment of the present disclosure.

[0137] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0138] Solution 1. A visual positioning method, comprising:

[0139] Obtaining the positioning information of the vehicle and the corresponding initial weights, and collecting images. The positioning information includes a first vehicle positioning determined by a visual positioning method and a second vehicle positioning determined by at least one other positioning method. Each vehicle positioning has a corresponding initial weight;

[0140] Obtaining an image quality score of the collected image, where the image quality score represents the proportion of the invalid area;

[0141] Reallocating the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight;

[0142] Performing positioning fusion according to the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain a fused positioning result of the vehicle.

[0143] Solution 2. The method according to Solution 1, wherein obtaining the image quality score of the collected image includes:

[0144] Inputting the collected image into a pre-trained image quality determination model to obtain an image quality score. The image quality determination model is trained from an initial neural network model based on sample images and the proportion annotations of the invalid areas corresponding to the sample images.

[0145] Solution 3. The method according to Solution 2, wherein the image quality determination model includes a feature extraction module, an attention module, and an output module. The feature extraction module is used to extract image features, the attention module is used to enhance the weight of the invalid area of the image, and the output module is used to output an image quality score determined according to the proportion of the invalid area.

[0146] Solution 4. The method according to any one of Solutions 1-3, wherein the invalid area represents an area with a small change in pixel value, and the invalid area includes at least one of an overexposed area, an underexposed area, and a low-texture area.

[0147] Solution 5. According to the method described in Solution 1, reallocating the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight, including:

[0148] Raising or lowering the initial weight of the first vehicle positioning according to the magnitude of the image quality score to obtain a reallocated weight, wherein the reallocated weight of the first vehicle positioning is inversely proportional to the image quality score, and the larger the image quality score, the higher the proportion of the invalid area is characterized.

[0149] Solution 6. According to the method described in Solution 5, the reallocated weight is equal to the product of the initial weight and the target score, the target score is the difference between 1 and the image quality score, and the value of the image quality score is between 0 and 1.

[0150] Solution 7. According to the method described in Solution 1, the first vehicle positioning determined by the visual positioning method includes:

[0151] Extracting a plurality of real-time visual features and a plurality of existing map visual features from the acquired image, and the map visual features have corresponding confidence levels;

[0152] Matching the plurality of real-time visual features among the plurality of map visual features to determine the first real-time visual feature with successful matching;

[0153] Based on the first real-time visual feature, the first map visual feature matched with the first real-time visual feature, the confidence level of the first map visual feature, and the error calculation function, determining the vehicle positioning with the smallest error as the first vehicle positioning.

[0154] Solution 8. A visual positioning and mapping method, characterized by including:

[0155] Obtaining the acquired image of the vehicle;

[0156] Obtaining the image quality score of the acquired image, and the image quality score characterizes the proportion of the invalid area;

[0157] Extracting a plurality of real-time visual features and a plurality of existing map visual features from the acquired image, and matching the plurality of real-time visual features among the plurality of map visual features to determine the second real-time visual feature with failed matching;

[0158] Setting the confidence level of the second real-time visual feature according to the image quality score, wherein the higher the image quality score, the lower the confidence level;

[0159] Based on the fusion positioning result of the vehicle, the second real-time visual feature, and its confidence level, inputting into the visual mapping module to create a visual map.

[0160] Solution 9. The method according to Solution 8, wherein the confidence level is equal to the difference between 1 and the image quality score, and the value of the confidence level ranges from 0 to 1.

[0161] Solution 10. The method according to Solution 8, wherein the image quality score is obtained by inputting the acquired image into a pre-trained image quality determination model.

[0162] Solution 11. The method according to Solution 8, wherein the fusion positioning result of the vehicle is obtained by performing positioning fusion on a third vehicle positioning determined by using a visual positioning method and a fourth vehicle positioning determined by at least one other positioning method;

[0163] Determining the third vehicle positioning by using a visual positioning method includes:

[0164] Obtaining a successfully matched third real-time visual feature;

[0165] Based on the third real-time visual feature, determining the third vehicle positioning by using a visual positioning method.

[0166] Solution 12. The method according to Solution 11, wherein determining the third vehicle positioning by using a visual positioning method based on the third real-time visual feature includes:

[0167] Based on the third real-time visual feature, a second map visual feature matched with the third real-time visual feature, the confidence level of the second map visual feature, and an error calculation function, determining the vehicle positioning with the minimum error as the third vehicle positioning, wherein the confidence level of the second map visual feature is set based on a historical image quality score.

[0168] Solution 13. The method according to Solution 8, the method further includes:

[0169] After the visual map is created, determining the fusion positioning result of the vehicle based on the created visual map.

[0170] Solution 14. A visual positioning device, including:

[0171] An acquisition module, configured to acquire the positioning information of the vehicle, the corresponding initial weight, and the acquired image, wherein the positioning information includes a first vehicle positioning determined by using a visual positioning method and a second vehicle positioning determined by at least one other positioning method, and each vehicle positioning has a corresponding initial weight;

[0172] An image quality module, configured to acquire the image quality score of the acquired image, and the image quality score represents the proportion of the invalid area;

[0173] A weight distribution module, configured to re - distribute the initial weight of the first vehicle positioning according to the image quality score to obtain a re - distributed weight;

[0174] A positioning module, configured to perform positioning fusion according to the re - distributed weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain a fused positioning result of the vehicle.

[0175] Solution 15. A visual positioning and mapping device, comprising:

[0176] An image module, configured to acquire a captured image of a vehicle;

[0177] A quality module, configured to acquire an image quality score of the captured image, where the image quality score represents the proportion of the invalid area;

[0178] A matching failure module, configured to extract a plurality of real - time visual features and a plurality of existing map visual features from the captured image, and match the plurality of real - time visual features in the plurality of map visual features to determine a second real - time visual feature with a matching failure;

[0179] A confidence module, configured to set the confidence of the second real - time visual feature according to the image quality score, where the higher the image quality score, the lower the confidence;

[0180] A map module, configured to create a visual map based on the fused positioning result of the vehicle, the second real - time visual feature, and its confidence input to a visual mapping module.

[0181] Solution 16. An electronic device, the electronic device comprising:

[0182] A processor;

[0183] A memory for storing executable instructions executable by the processor;

[0184] The processor is configured to read the executable instructions from the memory and execute the instructions to implement any one of the visual positioning methods in Solutions 1 - 7 above, or any one of the visual positioning and mapping methods in Solutions 8 - 13 above.

[0185] Solution 17. A computer - readable storage medium, the storage medium storing a computer program, where the computer program is used to execute any one of the visual positioning methods in Solutions 1 - 7 above, or any one of the visual positioning and mapping methods in Solutions 8 - 13 above.

[0186] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0187] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual positioning method, characterized in that, Including: Obtain the positioning information of the vehicle and the corresponding initial weights, and collect images. The positioning information includes a first vehicle positioning determined by a visual positioning method and a second vehicle positioning determined by at least one other positioning method. Each vehicle positioning has a corresponding initial weight; Obtain the image quality score of the collected image, where the image quality score represents the proportion of the invalid area; Reassign the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight; Perform positioning fusion based on the reallocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fused positioning result of the vehicle; The first vehicle positioning determined by the visual positioning method includes: Extract a plurality of real-time visual features and a plurality of existing map visual features from the collected image. The map visual features have corresponding confidence levels; Match the plurality of real-time visual features among the plurality of map visual features to determine the first real-time visual feature with successful matching; Based on the first real-time visual feature, the first map visual feature matched with the first real-time visual feature, the confidence level of the first map visual feature, and an error calculation function, determine the vehicle positioning with the minimum error as the first vehicle positioning.

2. The method according to claim 1, wherein Obtain the image quality score of the collected image, including: Input the collected image into a pre-trained image quality determination model to obtain an image quality score. The image quality determination model is trained based on a neural network initial model using sample images and the proportion annotations of the invalid areas corresponding to the sample images.

3. The method according to claim 2, characterized in that, The image quality determination model includes a feature extraction module, an attention module, and an output module. The feature extraction module is used to extract image features. The attention module is used to enhance the weight of the invalid area of the image. The output module is used to output the image quality score determined according to the proportion of the invalid area.

4. The method according to any one of claims 1 to 3, characterized in that The invalid area represents an area with relatively small pixel value changes, and the invalid area includes at least one of an overexposed area, an underexposed area, and a low-texture area.

5. The method according to claim 1, characterized in that Reassign the initial weight of the first vehicle positioning according to the image quality score to obtain a reallocated weight, including: Increase or decrease the initial weight of the first vehicle positioning according to the size of the image quality score to obtain a reallocated weight. Among them, the reallocated weight of the first vehicle positioning is inversely proportional to the image quality score, and the larger the image quality score, the higher the proportion of the invalid area.

6. The method according to claim 5, wherein The reallocated weight is equal to the product of the initial weight and the target score. The target score is the difference between 1 and the image quality score, and the value of the image quality score ranges from 0 to 1.

7. A visual positioning and mapping method, characterized in that Including: Obtain the collected image of the vehicle; Obtain the image quality score of the collected image, where the image quality score represents the proportion of the invalid area; Extract a plurality of real-time visual features and a plurality of existing map visual features from the collected image, and match the plurality of real-time visual features among the plurality of map visual features to determine the second real-time visual feature with failed matching; Set the confidence level of the second real-time visual feature according to the image quality score, where the higher the image quality score, the lower the confidence level; Based on the fusion positioning result of the vehicle, the second real-time visual feature, and its confidence level, input them into the visual mapping module to create a visual map.

8. The method according to claim 7, wherein The confidence level is equal to the difference between 1 and the image quality score, and the value of the confidence level is between 0 and 1.

9. The method according to claim 7, wherein The image quality score is obtained by inputting the acquired image into a pre-trained image quality determination model.

10. The method according to claim 7, wherein The fusion positioning result of the vehicle is obtained by performing positioning fusion on the third vehicle positioning determined by using the visual positioning method and the fourth vehicle positioning determined by at least one other positioning method; Determining the third vehicle positioning by using the visual positioning method includes: Obtain the successfully matched third real-time visual feature; Based on the third real-time visual feature, use the visual positioning method to determine the third vehicle positioning.

11. The method according to claim 10, wherein, Based on the third real-time visual feature, using the visual positioning method to determine the third vehicle positioning includes: Based on the third real-time visual feature, the second map visual feature matched with the third real-time visual feature, the confidence level of the second map visual feature, and the error calculation function, determine that the vehicle positioning with the minimum error is the third vehicle positioning, where the confidence level of the second map visual feature is set based on the historical image quality score.

12. The method according to claim 7, wherein The method further includes: After the visual map is created, determine the fusion positioning result of the vehicle based on the created visual map.

13. A visual positioning device, characterized in that, Includes: An acquisition module, configured to acquire the positioning information of the vehicle, the corresponding initial weight, and the acquired image. The positioning information includes the first vehicle positioning determined by using the visual positioning method and the second vehicle positioning determined by at least one other positioning method. Each vehicle positioning has a corresponding initial weight; An image quality module, configured to obtain the image quality score of the acquired image, where the image quality score represents the proportion of the invalid area; A weight allocation module, configured to re-allocate the initial weight of the first vehicle positioning according to the image quality score to obtain a re-allocated weight; A positioning module, configured to perform positioning fusion according to the re-allocated weight of the first vehicle positioning and the initial weight of the second vehicle positioning to obtain the fusion positioning result of the vehicle; The device further includes a first positioning module, configured to: extract a plurality of real-time visual features and a plurality of existing map visual features from the acquired image. The map visual features have corresponding confidence levels; match the plurality of real-time visual features in the plurality of map visual features to determine the successfully matched first real-time visual feature; Based on the first real-time visual feature, the first map visual feature matched with the first real-time visual feature, the confidence level of the first map visual feature, and the error calculation function, determine that the vehicle positioning with the minimum error is the first vehicle positioning.

14. A visual positioning and mapping device, characterized in that, Includes: An image module, configured to acquire the acquired image of the vehicle; A quality module, configured to obtain the image quality score of the acquired image, where the image quality score represents the proportion of the invalid area; A matching failure module, configured to extract a plurality of real-time visual features and a plurality of existing map visual features from the collected image, match the plurality of real-time visual features among the plurality of map visual features, and determine a second real-time visual feature with a matching failure; A confidence module, configured to set the confidence of the second real-time visual feature according to the image quality score, wherein the higher the image quality score, the lower the confidence; A map module, configured to create a visual map based on the fusion positioning result of the vehicle, the second real-time visual feature and its confidence input to a visual mapping module.

15. An electronic device, characterized in that, The electronic device includes: A processor; A memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the visual positioning method according to any one of claims 1-6 above, or the visual positioning and mapping method according to any one of claims 7-12 above.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is configured to execute the visual positioning method according to any one of claims 1-6 above, or the visual positioning and mapping method according to any one of claims 7-12 above.

Images