Navigation method, device, electronic device and storage medium based on image analysis

Abstract

The present invention relates to the field of navigation technology, and in particular to a navigation method, device, electronic device and storage medium based on image analysis. The method of the present invention first obtains a navigation route, and then extracts the characteristic size of a target feature and the position of the target feature in the first image from a first image based on position estimation and first posture information; then the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position are input into a positioning model, and the first position is determined based on the second position and the relative position output by the positioning model; finally, based on the second position and the first position, the target point is found from the navigation route, and the driving force and the direction of travel are adjusted according to the target point. The implementation method of the present invention is based on image feature positioning. Since the entity position corresponding to the image feature is fixed, when positioning is performed based on the image of the entity, the cumulative error is small and the positioning accuracy is high.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to a navigation method, device, electronic equipment and storage medium based on image analysis. Background Art

[0002] The power survey robot is a robot used for power survey. One of its application scenarios is to survey underground pipe corridors to provide a basis for the upgrading and reconstruction of underground pipe corridors. In the related technology, an inertial navigation sensor is provided on the maintenance robot, and the data collected by the inertial navigation sensor is used to provide navigation for the robot, and to locate and mark the key points of the underground pipe corridor.

[0003] The inertial navigation system has the characteristic of large cumulative errors. When using the inertial navigation system to provide navigation, key point positioning and marking, as time goes by, the error will become larger and affect the final survey results.

[0004] Based on this, it is necessary to develop and design a navigation method based on image analysis. Summary of the invention

[0005] The embodiments of the present invention provide a navigation method, device, electronic device and storage medium based on image analysis, which are used to solve the problem of large navigation error in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a navigation method based on image analysis, comprising:

[0007] Get navigation route;

[0008] Extracting, from the first image, a feature size of a target feature and a position of the target feature in the first image as a first feature size and a first feature position according to the position estimation and the first posture information;

[0009] Input the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determine the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected;

[0010] According to the second position and the first position, a target point that meets the travel driving force constraint and the travel direction constraint is found on the navigation route, and the travel driving force and the travel direction are adjusted according to the target point.

[0011] In a possible implementation, extracting a feature size of a target feature and a position of the target feature in the first image from the first image according to the position estimation and the first posture information as a first feature size and a first feature position includes:

[0012] Performing edge processing on the first image to obtain an edge feature map;

[0013] calculating a distance between the estimated position and the second position as a first distance;

[0014] Calculating a ratio of the first distance to the second distance as a distance ratio, wherein the distance is a distance between a position at which the second image is acquired and a position at which the third image is acquired;

[0015] adjusting the size of the target feature sample according to the distance ratio;

[0016] Rotate the resized target feature sample according to the first posture information to obtain a feature sample to be compared;

[0017] Extracting an image block from the edge feature map in a sliding manner and comparing the image block with the feature sample to be compared for similarity, finding the position of the most similar image block in the edge feature map as the first feature position;

[0018] The size of the feature sample to be compared is gradually adjusted, and after each adjustment, a similarity comparison is performed with the image block at the first feature position, and the size of the most similar feature sample to be compared is found from multiple comparisons as the first feature size.

[0019] In a possible implementation, the method of extracting an image block from the edge feature map by sliding and comparing the image block with the feature sample to be compared to find the position of the most similar image block in the edge feature map as the first feature position includes:

[0020] Get and initialize the position index;

[0021] Extracting, from the edge feature map according to the position index, an image block of the same type as the feature sample to be compared as the image block to be compared;

[0022] The similarity coefficient is determined according to a first formula, the image block to be compared and the feature sample to be compared, wherein the first formula is:

[0023]

[0024] In the formula, is the similarity coefficient, is the image block to be compared Line The pixel value of the column, is the feature sample to be compared Line The pixel value of the column, is the total number of columns of feature samples to be compared, is the total number of rows of feature samples to be compared;

[0025] adding the similarity coefficient to a similarity coefficient queue;

[0026] If the traversal of the edge feature map is not completed, the position index is offset, and the process jumps to the step of extracting an image block of the same type as the feature sample to be compared from the edge feature map according to the position index as the image block to be compared;

[0027] Selecting the similarity coefficient with the largest value from the similarity coefficient queue as the target coefficient;

[0028] The position of the image block corresponding to the target coefficient in the edge feature map is used as the first feature position;

[0029] The adjusting the size of the target feature sample according to the distance ratio comprises:

[0030] The size of the target feature sample is adjusted according to a second formula, the distance ratio, and a size ratio of the target feature in the third image to the target feature in the second image, wherein the second formula is:

[0031]

[0032] In the formula, is the size of the adjusted target feature sample, is the distance ratio, is the size ratio of the target feature in the third image to the target feature in the second image, is the size of the target feature sample.

[0033] In a possible implementation, the process of constructing the positioning model includes:

[0034] Acquire multiple sample data sets, wherein the sample data sets include: first posture information sample data, first feature size sample data, first feature position sample data, second posture information sample data, second feature size sample data and second feature position sample data, and each sample data set corresponds to a relative position label;

[0035] traversally extracting a sample data set from the plurality of sample data sets as a data set to be input;

[0036] Inputting the data set to be input into a positioning basic model to obtain a relative position estimation value output by the positioning basic model;

[0037] Inputting the relative position estimation value and the data set to be input into a loss calculation function to obtain an estimated loss;

[0038] If the estimated loss is greater than the loss threshold, a gradient descent method is used to adjust multiple parameters of the positioning basic model from the node on the output side to the node on the input side, and the process jumps to the step of traversally extracting a sample data set from the multiple sample data sets as the data set to be input;

[0039] Otherwise, the positioning basic model is used as the positioning model.

[0040] In a possible implementation, the positioning basic model is:

[0041]

[0042] In the formula, The relative position of Dimensional elements, is the total number of rows of intermediate nodes, For the The output node weights, For the Line The first column node weights, For the Line The output of the middle node of the column, is the th input variable, is the total number of input variables, is the total number of columns of intermediate nodes, is a natural constant, For the The output bias, For the Line Column node bias.

[0043] In a possible implementation, the step of finding a target point on the navigation route that meets the driving force constraint and the traveling direction constraint according to the second position and the first position, and adjusting the driving force and the traveling direction according to the target point includes:

[0044] Determine a first travel distance and a first travel direction according to the second position and the first position;

[0045] Planning a first circle with the first position as the center and the first travel distance as the radius;

[0046] Finding a plurality of intersection points with the first circle from the navigation route;

[0047] using the travel directions from the first position to the plurality of intersections as a plurality of first directions;

[0048] taking differences between the plurality of first directions and the traveling direction as a plurality of direction differences;

[0049] According to the multiple direction differences, searching for an intersection point satisfying the travel direction constraint from the multiple intersection points;

[0050] If there is no intersection satisfying the travel direction constraint, adjusting the first travel distance according to the travel driving force constraint, and jumping to the step of planning a first circle with the first position as the center and the first travel distance as the radius;

[0051] Otherwise, the intersection point satisfying the travel direction constraint is taken as the target point;

[0052] The driving force and the direction of travel are adjusted according to the target point and the first position.

[0053] In a possible implementation manner, adjusting the driving force and the traveling direction according to the target point and the first position includes:

[0054] Determine a target moving speed and a target moving direction according to the target point, the first position, and a time node of reaching the target point;

[0055] adjusting the direction of travel according to the target direction of travel;

[0056] The driving force is adjusted according to a third formula, the target traveling speed and the current traveling speed, wherein the third formula is:

[0057]

[0058] In the formula, As driving force, is the difference between the target speed and the current speed, , as well as They are the first weight, the second weight and the third weight respectively.

[0059] In a second aspect, an embodiment of the present invention provides a navigation device based on image analysis, which is used to implement the navigation method based on image analysis as described in the first aspect or any possible implementation of the first aspect, and the navigation device based on image analysis includes:

[0060] A navigation route acquisition module is used to obtain a navigation route;

[0061] An image feature extraction module, configured to extract a feature size of a target feature and a position of the target feature in the first image from the first image as a first feature size and a first feature position according to the position estimation and the first posture information;

[0062] A positioning module, used for inputting the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determining the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and the characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected;

[0063] as well as,

[0064] The navigation module is used to find a target point that meets the driving force constraint and the traveling direction constraint on the navigation route according to the second position and the first position, and adjust the driving force and the traveling direction according to the target point.

[0065] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation method of the first aspect.

[0066] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0067] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0068] The embodiment of the present invention discloses a navigation method based on image analysis, which first obtains a navigation route; then extracts the characteristic size of a target feature and the position of the target feature in the first image from the first image according to the position estimation and the first posture information, as the first characteristic size and the first characteristic position; then inputs the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determines the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and the characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected; finally, according to the second position and the first position, finds a target point on the navigation route that meets the driving force constraint and the driving direction constraint, and adjusts the driving force and the driving direction according to the target point. The embodiment of the present invention locates the current position based on the comparison of the target characteristic size and position in the two images before and after. Since the entity position corresponding to the image feature is fixed, when positioning is performed based on the image of the entity, the cumulative error is small and the positioning accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0070] Figure 1 is a flow chart of a navigation method based on image analysis provided by an embodiment of the present invention;

[0071] Figure 2 is a schematic diagram of the first characteristic position of the sliding positioning provided by an embodiment of the present invention;

[0072] Figure 3 It is a schematic diagram of the basic positioning model structure provided by an embodiment of the present invention;

[0073] Figure 4 It is a principle diagram of the process of finding a target point from a navigation route provided by an embodiment of the present invention;

[0074] Figure 5 is a functional block diagram of a navigation device based on image analysis provided by an embodiment of the present invention;

[0075] Figure 6 It is a functional block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present invention with unnecessary details.

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clear, a specific implementation method will be described below in conjunction with the accompanying drawings.

[0078] The following is a detailed description of an embodiment of the present invention. This example is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0079] Figure 1 A flowchart of a navigation method based on image analysis provided in an embodiment of the present invention.

[0080] like Figure 1 As shown, it shows a flowchart of the implementation of the navigation method based on image analysis provided by an embodiment of the present invention, which is described in detail as follows:

[0081] In step 101, a navigation route is obtained.

[0082] In step 102, based on the position estimation and the first posture information, a feature size of a target feature and a position of the target feature in the first image are extracted from the first image as a first feature size and a first feature position.

[0083] In some implementations, step 102 includes:

[0084] The step of extracting a feature size of a target feature and a position of the target feature in the first image from the first image according to the position estimation and the first posture information as a first feature size and a first feature position includes:

[0085] Performing edge processing on the first image to obtain an edge feature map;

[0086] calculating a distance between the estimated position and the second position as a first distance;

[0087] Calculating a ratio of the first distance to the second distance as a distance ratio, wherein the distance is a distance between a position at which the second image is acquired and a position at which the third image is acquired;

[0088] adjusting the size of the target feature sample according to the distance ratio;

[0089] Rotate the resized target feature sample according to the first posture information to obtain a feature sample to be compared;

[0090] Extracting an image block from the edge feature map in a sliding manner and comparing the image block with the feature sample to be compared for similarity, finding the position of the most similar image block in the edge feature map as the first feature position;

[0091] The size of the feature sample to be compared is gradually adjusted, and after each adjustment, a similarity comparison is performed with the image block at the first feature position, and the size of the most similar feature sample to be compared is found from multiple comparisons as the first feature size.

[0092] In some embodiments, the method of extracting an image block from the edge feature map by sliding and comparing the image block with the feature sample to be compared to find the position of the most similar image block in the edge feature map as the first feature position includes:

[0093] Get and initialize the position index;

[0094] Extracting, from the edge feature map according to the position index, an image block of the same type as the feature sample to be compared as the image block to be compared;

[0095] The similarity coefficient is determined according to a first formula, the image block to be compared and the feature sample to be compared, wherein the first formula is:

[0096]

[0097] In the formula, is the similarity coefficient, is the image block to be compared Line The pixel value of the column, is the feature sample to be compared Line The pixel value of the column, is the total number of columns of feature samples to be compared, is the total number of rows of feature samples to be compared;

[0098] adding the similarity coefficient to a similarity coefficient queue;

[0099] If the traversal of the edge feature map is not completed, the position index is offset, and the process jumps to the step of extracting an image block of the same type as the feature sample to be compared from the edge feature map according to the position index as the image block to be compared;

[0100] Selecting the similarity coefficient with the largest value from the similarity coefficient queue as the target coefficient;

[0101] The position of the image block corresponding to the target coefficient in the edge feature map is used as the first feature position;

[0102] The adjusting the size of the target feature sample according to the distance ratio comprises:

[0103] The size of the target feature sample is adjusted according to a second formula, the distance ratio, and a size ratio of the target feature in the third image to the target feature in the second image, wherein the second formula is:

[0104]

[0105] In the formula, is the size of the adjusted target feature sample, is the distance ratio, is the size ratio of the target feature in the third image to the target feature in the second image, is the size of the target feature sample.

[0106] Exemplarily, as mentioned above, in the prior art, the current position is located by an inertial navigation system, and an indication of the direction and speed of travel is issued based on the navigation route. The advantage of this system is that it can be less dependent on the environment and is suitable for application scenarios lacking navigation satellite signals. The disadvantage is that the cumulative error is large, and as time goes by, the positioning error will gradually increase. In order to overcome the disadvantage of the large cumulative error of the inertial navigation system, the embodiment of the present invention extracts features from the image based on the attitude information obtained by the inertial navigation system and the position estimation made according to the inertial navigation system, and determines the relative position of the image taken later relative to the image taken earlier through the position and size of the features in two images taken successively. After determining the position, find the target point that meets the travel capability of the device from the navigation route (the point when the next image is taken), and adjust the driving force and travel direction of the device according to the target point.

[0107] To achieve the above-mentioned purpose, the implementation mode of the present invention first performs edge processing on the latest image. There are many ways of edge processing. For example, in some scenarios, the image is first decolorized and blurred, and then the canny operator is used to filter the image, followed by non-maximum suppression, and finally a threshold is set to perform binarization processing to obtain the edge of the image.

[0108] For another example, in some scenes, the inverse color image of the original image is used for color fading processing, and the edge image can be directly obtained. The fading process of each pixel can be expressed by the following formula:

[0109]

[0110] In the formula, To fade the The pixel value of the position, In grayscale The pixel value of the position, Inverted color map The pixel value at the location.

[0111] In terms of the production of target feature samples, an implementation of the present invention calculates the ratio of the distance traveled from the previous time node to the current time node and the distance from the previous two time nodes to the previous time node, scales the target feature samples according to this ratio, and then rotates the target feature samples according to the attitude information fed back by the inertial navigation system to obtain the final feature samples to be compared.

[0112] The embodiment of the present invention locates the position of the feature sample to be compared in the edge feature map by sliding, and adjusts the size of the feature sample to be compared by different scaling ratios to determine the size of the feature sample to be compared in the edge feature map.

[0113] Figure 2 A method for determining the position of the feature sample to be compared in the edge feature map by sliding is provided. First, an image block 202 of the same type as the feature sample to be compared is extracted from the edge feature map 201. The extracted image block 202 and the feature sample to be compared 203 are calculated using a first formula for similarity coefficient S. The first formula is:

[0114]

[0115] In the formula, is the similarity coefficient, is the image block to be compared Line The pixel value of the column, is the feature sample to be compared Line The pixel value of the column, is the total number of columns of feature samples to be compared, is the total number of rows of feature samples to be compared;

[0116] The calculated similarity coefficient S is added to the queue, and then the position of the image block 202 is slid (from left to right and from top to bottom in the figure), the image block is taken out again and the similarity coefficient is calculated until the edge feature map 201 is traversed. After the traversal is completed, the position of the maximum value is found from the similarity coefficient queue, which is the position of the target feature 204 in the edge feature map 201. After the position is determined, the feature sample 203 to be compared is scaled. After each scaling, a similarity comparison is performed with the image block at the target position. When the similarity is maximum, the size of the feature sample 203 to be compared corresponding to the scaling ratio is the size of the target feature in the edge feature map 201.

[0117] In step 103, the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position are input into the positioning model, and the first position is determined according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are respectively the characteristic size and the characteristic position of the target feature in the second image, and the second position is the position when the second image is acquired.

[0118] In some implementations, step 103 includes:

[0119] The construction process of the positioning model includes:

[0120] Acquire multiple sample data sets, wherein the sample data sets include: first posture information sample data, first feature size sample data, first feature position sample data, second posture information sample data, second feature size sample data and second feature position sample data, and each sample data set corresponds to a relative position label;

[0121] traversally extracting a sample data set from the plurality of sample data sets as a data set to be input;

[0122] Inputting the data set to be input into a positioning basic model to obtain a relative position estimation value output by the positioning basic model;

[0123] Inputting the relative position estimation value and the data set to be input into a loss calculation function to obtain an estimated loss;

[0124] If the estimated loss is greater than the loss threshold, a gradient descent method is used to adjust multiple parameters of the positioning basic model from the node on the output side to the node on the input side, and the process jumps to the step of traversally extracting a sample data set from the multiple sample data sets as the data set to be input;

[0125] Otherwise, the positioning basic model is used as the positioning model.

[0126] In some implementations, the positioning base model is:

[0127]

[0128] In the formula, The relative position of Dimensional elements, is the total number of rows of intermediate nodes, For the The output node weights, For the Line The first column node weights, For the Line The output of the middle node of the column, is the th input variable, is the total number of input variables, is the total number of columns of intermediate nodes, is a natural constant, For the The output bias, For the Line Column node bias.

[0129] Exemplarily, an embodiment of the present invention inputs the posture information of the current time node, the characteristic size of the target feature in the image, the characteristic position, and the posture information of the previous time node, the characteristic size of the target feature in the image, and the characteristic position into a positioning model. The model will give an indication of a relative position based on this information, and the model is constructed based on a sample data set and a positioning basic model.

[0130] Figure 3 The overall structure of the positioning basic model is given. In the figure, the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position are input into the input node 301 of the positioning basic model as input quantities, and are propagated backward in sequence through the intermediate node 302 until reaching the output node 303, and the indication of the relative position is output at the output node 303. The above positioning basic model is expressed by the formula:

[0131]

[0132] In the formula, The relative position of Dimensional elements, is the total number of rows of intermediate nodes, For the The output node weights, For the Line The first column node weights, For the Line The output of the middle node of the column, is the th input variable, is the total number of input variables, is the total number of columns of intermediate nodes, is a natural constant, For the The output bias, For the Line Column node bias.

[0133] We can see that there are multiple weight parameters in the formula, and whether these parameters are set reasonably is the key to the success of the model construction. In the aspect of using the positioning basic model to construct the positioning model, the embodiment of the present invention uses multiple sample data sets, each of which includes: first posture information sample data, first feature size sample data, first feature position sample data, second posture information sample data, second feature size sample data and second feature position sample data. In addition, each data set is also provided with a label corresponding to the relative position of the above data.

[0134] Extract the data sets from the above data sets in turn and input them into the positioning basic model. The positioning basic model will output an estimated value of the relative position. This estimated value and the label of the relative position are input into the loss calculation function to obtain the estimated loss. The loss calculation function can be a mean square error function or a cross entropy function. These functions will give the estimated loss of the positioning basic model based on the estimated value and the label. If the estimated loss is greater than the loss threshold, the gradient descent method is used from the nodes on the output side to the nodes on the input side. According to the estimated loss, the gradient descent method is used to adjust multiple parameters of the positioning basic model. When the estimated loss calculated by the loss function after multiple sample data set inputs is less than the threshold, the parameters of the positioning basic model can be fixed and the positioning basic model can be used as the positioning basic model.

[0135] In step 104, according to the second position and the first position, a target point that meets the driving force constraint and the traveling direction constraint is found on the navigation route, and the driving force and the traveling direction are adjusted according to the target point.

[0136] In some implementations, step 104 includes:

[0137] The step of finding a target point on the navigation route that meets the driving force constraint and the traveling direction constraint according to the second position and the first position, and adjusting the driving force and the traveling direction according to the target point includes:

[0138] Determine a first travel distance and a first travel direction according to the second position and the first position;

[0139] Planning a first circle with the first position as the center and the first travel distance as the radius;

[0140] Finding a plurality of intersection points with the first circle from the navigation route;

[0141] using the travel directions from the first position to the plurality of intersections as a plurality of first directions;

[0142] taking differences between the plurality of first directions and the traveling direction as a plurality of direction differences;

[0143] According to the multiple direction differences, searching for an intersection point satisfying the travel direction constraint from the multiple intersection points;

[0144] If there is no intersection satisfying the travel direction constraint, adjusting the first travel distance according to the travel driving force constraint, and jumping to the step of planning a first circle with the first position as the center and the first travel distance as the radius;

[0145] Otherwise, the intersection point satisfying the travel direction constraint is taken as the target point;

[0146] The driving force and the direction of travel are adjusted according to the target point and the first position.

[0147] In some embodiments, adjusting the driving force and the direction of travel according to the target point and the first position includes:

[0148] Determine a target moving speed and a target moving direction according to the target point, the first position, and a time node of reaching the target point;

[0149] adjusting the direction of travel according to the target direction of travel;

[0150] The driving force is adjusted according to a third formula, the target traveling speed and the current traveling speed, wherein the third formula is:

[0151]

[0152] In the formula, As driving force, is the difference between the target speed and the current speed, , as well as They are the first weight, the second weight and the third weight respectively.

[0153] For example, Figure 4 As shown, the embodiment of the present invention takes the distance 404 between the current position 402 and the position 403 of the previous time node as the radius 405, and takes the current position 402 as the center to plan a circle, and the intersection of this circle and the navigation route 401 is the point on the navigation route 401 traveled at the current travel speed (in some cases, there will be two intersections). If the direction angle from the current position 402 to this point is significantly different from the direction angle from the position 403 of the previous time node to the current position 402, it means that the travel direction will change significantly in a short period of time and adjustments should be made. The embodiment of the present invention re-plans the radius based on the change in travel speed to form a new radius 406. After the new radius 406 is generated, the above steps of finding the target point are repeated.

[0154] After the target point is determined, the direction can be adjusted to the target direction, and then the driving force is adjusted according to the current speed and the target speed (the target speed is determined according to the next time node reaching the target node). The embodiment of the present invention adopts the third formula for adjustment:

[0155]

[0156] In the formula, As driving force, is the difference between the target speed and the current speed, , as well as They are the first weight, the second weight and the third weight respectively.

[0157] The embodiment of the navigation method based on image analysis of the present invention first obtains the navigation route; then extracts the characteristic size of the target feature and the position of the target feature in the first image from the first image according to the position estimation and the first posture information as the first characteristic size and the first characteristic position; then inputs the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into the positioning model, and determines the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and the characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected; finally, according to the second position and the first position, finds the target point that meets the driving force constraint and the driving direction constraint from the navigation route, and adjusts the driving force and the driving direction according to the target point. The embodiment of the present invention locates the current position based on the comparison of the target characteristic size and position in the two images before and after. Since the entity position corresponding to the image feature is fixed, when positioning is performed based on the image of the entity, the cumulative error is small and the positioning accuracy is high.

[0158] It should be understood that the size of the serial numbers of the steps in the above implementation does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation method of the present invention.

[0159] The following is an implementation of the device of the present invention. For details not described in detail, reference may be made to the corresponding method implementation described above.

[0160] Figure 5 is a functional block diagram of a navigation device based on image analysis provided by an embodiment of the present invention, referring to Figure 5 The navigation device based on image analysis includes: a navigation route acquisition module 501, an image feature extraction module 502, a positioning module 503 and a navigation module 504, wherein:

[0161] A navigation route acquisition module 501 is used to acquire a navigation route;

[0162] An image feature extraction module 502 is used to extract a feature size of a target feature and a position of the target feature in the first image from the first image as a first feature size and a first feature position according to the position estimation and the first posture information;

[0163] A positioning module 503, used for inputting the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determining the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected;

[0164] The navigation module 504 is used to find a target point that meets the driving force constraint and the traveling direction constraint on the navigation route according to the second position and the first position, and adjust the driving force and the traveling direction according to the target point.

[0165] Figure 6 is a functional block diagram of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, the electronic device 6 of this embodiment includes: a processor 600 and a memory 601, wherein the memory 601 stores a computer program 602 that can be run on the processor 600. When the processor 600 executes the computer program 602, the steps in the above-mentioned navigation method based on image analysis and the embodiments are implemented, for example Figure 1 Steps 101 to 104 are shown.

[0166] Exemplarily, the computer program 602 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 601 and executed by the processor 600 to implement the present invention.

[0167] The electronic device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 6 may include, but is not limited to, a processor 600 and a memory 601. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 6 may also include input and output devices, network access devices, buses, etc.

[0168] The processor 600 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0169] The memory 601 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 601 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 601 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 601 is used to store the computer program 602 and other programs and data required by the electronic device 6. The memory 601 may also be used to temporarily store data that has been output or is to be output.

[0170] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method implementation method, which will not be repeated here.

[0171] In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0175] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various methods and device implementation methods can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0177] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A navigation method based on image analysis, characterized in that: include: Get navigation route; Extracting, from the first image, a feature size of a target feature and a position of the target feature in the first image as a first feature size and a first feature position according to the position estimation and the first posture information; Input the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determine the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and characteristic position of the target feature in the second image, respectively, the second position is the position when the second image is collected, and the time node corresponding to the second image is earlier than the time node corresponding to the first image; Finding a target point on the navigation route that meets the travel driving force constraint and the travel direction constraint according to the second position and the first position, and adjusting the travel driving force and the travel direction according to the target point; in, The step of extracting a feature size of a target feature and a position of the target feature in the first image from the first image according to the position estimation and the first posture information as a first feature size and a first feature position includes: Performing edge processing on the first image to obtain an edge feature map; calculating a distance between the estimated position and the second position as a first distance; Calculating a ratio of the first distance to a second distance as a distance ratio, wherein the second distance is a distance between a position at which the second image is acquired and a position at which a third image is acquired, and a time node corresponding to the third image is earlier than a time node corresponding to the second image; adjusting the size of the target feature sample according to the distance ratio; Rotate the resized target feature sample according to the first posture information to obtain a feature sample to be compared; Extracting an image block from the edge feature map in a sliding manner and comparing the image block with the feature sample to be compared for similarity, finding the position of the most similar image block in the edge feature map as the first feature position; Step by step, adjusting the size of the feature sample to be compared, performing a similarity comparison with the image block at the first feature position after each adjustment, and finding the size of the most similar feature sample to be compared from multiple comparisons as the first feature size; in, The method of extracting an image block from the edge feature map by sliding and comparing the image block with the feature sample to be compared, finding the position of the most similar image block in the edge feature map as the first feature position, includes: Get and initialize the position index; Extracting, from the edge feature map according to the position index, an image block of the same type as the feature sample to be compared as the image block to be compared; The similarity coefficient is determined according to a first formula, the image block to be compared and the feature sample to be compared, wherein the first formula is: In the formula, is the similarity coefficient, is the image block to be compared Line The pixel value of the column, is the feature sample to be compared Line The pixel value of the column, is the total number of columns of feature samples to be compared, is the total number of rows of feature samples to be compared; adding the similarity coefficient to a similarity coefficient queue; If the traversal of the edge feature map is not completed, the position index is offset, and the process jumps to the step of extracting an image block of the same type as the feature sample to be compared from the edge feature map according to the position index as the image block to be compared; Selecting the similarity coefficient with the largest value from the similarity coefficient queue as the target coefficient; The position of the image block corresponding to the target coefficient in the edge feature map is used as the first feature position; The adjusting the size of the target feature sample according to the distance ratio comprises: The size of the target feature sample is adjusted according to a second formula, the distance ratio, and a size ratio of the target feature in the third image to the target feature in the second image, wherein the second formula is: In the formula, is the size of the adjusted target feature sample, is the distance ratio, is the size ratio of the target feature in the third image to the target feature in the second image, is the size of the target feature sample.

2. The navigation method based on image analysis according to claim 1, characterized in that: The construction process of the positioning model includes: Acquire multiple sample data sets, wherein the sample data sets include: first posture information sample data, first feature size sample data, first feature position sample data, second posture information sample data, second feature size sample data and second feature position sample data, and each sample data set corresponds to a relative position label; traversally extracting a sample data set from the plurality of sample data sets as a data set to be input; Inputting the data set to be input into a positioning basic model to obtain a relative position estimation value output by the positioning basic model; Inputting the relative position estimation value and the data set to be input into a loss calculation function to obtain an estimated loss; If the estimated loss is greater than the loss threshold, a gradient descent method is used to adjust multiple parameters of the positioning basic model from the node on the output side to the node on the input side, and the process jumps to the step of traversally extracting a sample data set from the multiple sample data sets as the data set to be input; Otherwise, the positioning basic model is used as the positioning model.

3. The navigation method based on image analysis according to claim 2, characterized in that: The positioning basic model is: In the formula, The relative position of Dimensional elements, is the total number of rows of intermediate nodes, For the The output node weights, For the Line The first column node weights, For the Line The output of the middle node of the column, is the th input variable, is the total number of input variables, is the total number of columns of intermediate nodes, is a natural constant, For the The output bias, For the Line Column node bias.

4. The navigation method based on image analysis according to any one of claims 1 to 3, characterized in that: The step of finding a target point on the navigation route that meets the driving force constraint and the traveling direction constraint according to the second position and the first position, and adjusting the driving force and the traveling direction according to the target point includes: Determine a first travel distance and a first travel direction according to the second position and the first position; Planning a first circle with the first position as the center and the first travel distance as the radius; Finding a plurality of intersection points with the first circle from the navigation route; using the travel directions from the first position to the plurality of intersections as a plurality of first directions; taking differences between the plurality of first directions and the traveling direction as a plurality of direction differences; According to the multiple direction differences, searching for an intersection point satisfying the travel direction constraint from the multiple intersection points; If there is no intersection satisfying the travel direction constraint, adjusting the first travel distance according to the travel driving force constraint, and jumping to the step of planning a first circle with the first position as the center and the first travel distance as the radius; Otherwise, the intersection point satisfying the travel direction constraint is taken as the target point; The driving force and the direction of travel are adjusted according to the target point and the first position.

5. The navigation method based on image analysis according to claim 4, characterized in that: The adjusting the driving force and the traveling direction according to the target point and the first position includes: Determine a target moving speed and a target moving direction according to the target point, the first position, and a time node of reaching the target point; adjusting the direction of travel according to the target direction of travel; The driving force is adjusted according to a third formula, the target traveling speed and the current traveling speed, wherein the third formula is: In the formula, As driving force, is the difference between the target speed and the current speed, , as well as They are the first weight, the second weight and the third weight respectively.

6. A navigation device based on image analysis, characterized in that: Used to implement the navigation method based on image analysis according to any one of claims 1 to 5, the navigation device based on image analysis comprises: A navigation route acquisition module is used to obtain a navigation route; An image feature extraction module, configured to extract a feature size of a target feature and a position of the target feature in the first image from the first image as a first feature size and a first feature position according to the position estimation and the first posture information; A positioning module, used for inputting the first posture information, the first characteristic size, the first characteristic position, the second posture information, the second characteristic size and the second characteristic position into a positioning model, and determining the first position according to the second position and the relative position output by the positioning model, wherein the second characteristic size and the second characteristic position are the characteristic size and the characteristic position of the target feature in the second image, respectively, and the second position is the position when the second image is collected; as well as, The navigation module is used to find a target point that meets the driving force constraint and the traveling direction constraint on the navigation route according to the second position and the first position, and adjust the driving force and the traveling direction according to the target point.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images