Positioning method, apparatus, device, and storage medium

Abstract

The application provides a positioning method, device, equipment and storage medium, the method comprises: obtaining the position information change of each object in the environment image data in a preset time period; according to the position information change, the first stable object and the to-be-confirmed object are obtained by classifying each object; obtain the feature image of the to-be-confirmed object; based on the feature image of the to-be-confirmed object, whether the to-be-confirmed object is a preset stable target is judged; if yes, the to-be-confirmed object is determined as a second stable object; the feature information of the first stable object and the second stable object is extracted, and positioning is carried out based on the positioning mapping algorithm and the feature information. It can effectively prevent the missed detection of the target stable object, and then realize positioning according to the feature information of the target stable object, which can effectively prevent the problem of positioning failure or drift, and improve the positioning accuracy.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a positioning method, apparatus, device and storage medium. Background Art

[0002] Currently, visual simultaneous localization and mapping (VSLAM) positioning technology is commonly used for navigation and positioning. Due to the large changes in outdoor environments, VSLAM positioning technology cannot obtain effective image feature information, resulting in low positioning accuracy and prone to positioning failure or drift. Summary of the Invention

[0003] The present application provides a positioning method, apparatus, device and storage medium, which are intended to improve the accuracy of positioning. By classifying each object according to the change in position information of each object in environmental image data within a preset time period, a first stable object and a stable object to be confirmed in the environment are determined, and whether the stable object to be confirmed is a second stable object is further determined. This can effectively prevent the missed detection of the second stable object, and then realize positioning based on the characteristic information of the first stable object and the second stable object, which can effectively prevent the problem of positioning failure or drift.

[0004] In a first aspect, an embodiment of the present application provides a positioning method, the method comprising:

[0005] Acquire environmental image data within a preset time period, and determine a change in position information of each object in the environmental image data;

[0006] Classifying the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed;

[0007] Acquiring a characteristic image of the object to be confirmed;

[0008] determining, based on the characteristic image of the object to be confirmed, whether the object to be confirmed is a second stable object;

[0009] Feature information of the first stable object and the second stable object is extracted, and positioning is performed based on a positioning and mapping algorithm and the feature information.

[0010] In one embodiment, the classifying the target objects according to the change in the position information to obtain the first stable object and the object to be confirmed includes:

[0011] When the change in the position information of the target object is less than a first position change threshold, determining that the target object is a first stable object;

[0012] When the change in the position information of the target object is greater than or equal to the first position change threshold and less than the second position change threshold, the target object is determined to be an object to be confirmed.

[0013] In one embodiment, judging whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed includes:

[0014] Performing object classification detection on the object to be confirmed in the feature image to obtain the category of the object to be confirmed;

[0015] If the category of the object to be confirmed belongs to the preset category, determining that the object to be confirmed is a second stable object;

[0016] If the category of the object to be confirmed does not belong to the preset category, generating an object confirmation request based on the feature image of the object to be confirmed and sending it to the target device, wherein the object confirmation request is used to prompt a user on the target device whether to select the object to be confirmed as the second stable object based on the feature image of the object to be confirmed;

[0017] A selection result of the user is received, and whether the object to be confirmed is a second stable object is determined according to the selection result.

[0018] In one embodiment, the determining whether the object to be confirmed is a second stable object based on the feature image of the object to be confirmed further includes:

[0019] Acquiring prompt information, and sending the characteristic image and prompt information to a target device, wherein the prompt information is used to prompt a user on the target device whether to select the object to be confirmed as a second stable object based on the characteristic image of the object to be confirmed;

[0020] A selection result of the user is received, and whether the object to be confirmed is a second stable object is determined according to the selection result.

[0021] In one embodiment, determining the change in position information of each target object in the environmental image data includes:

[0022] Acquire first location information and second location information of the target object respectively, wherein the first location information is location information of the target object at a first preset moment within the preset time period, and the second location information is location information of the target object at a second preset moment within the preset time period;

[0023] A change in the location information of the target object within the preset time period is determined based on the first location information and the second location information.

[0024] In one embodiment, the method further comprises:

[0025] acquiring, from the environmental image data, a first image feature of a preset position at a third preset moment and a second image feature of a fourth preset moment;

[0026] Performing object recognition on the first image feature and the second image feature respectively to obtain a first recognized object and a second recognized object;

[0027] When the first recognition object and the second recognition object are the same, determining that a first stable object exists at the preset position;

[0028] When the first recognition object and the second recognition object are different, it is determined that there is an object to be confirmed at the preset position.

[0029] In one embodiment, the method further comprises:

[0030] Acquire a first image region corresponding to the first recognition object and a second image region corresponding to the second recognition object;

[0031] calculating an intersection-over-union ratio of the first image area and the second image area;

[0032] When the IoU meets a preset IoU threshold, it is determined that the first recognition object and the second recognition object are the same; otherwise, it is determined that the first recognition object and the second recognition object are not the same.

[0033] In a second aspect, an embodiment of the present application provides a positioning device, the device comprising:

[0034] A first acquisition module is used to acquire environmental image data within a preset time period and determine a change in position information of each target object in the environmental image data;

[0035] a classification module, configured to classify the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed;

[0036] A second acquisition module is used to acquire a characteristic image of the object to be confirmed;

[0037] a judgment module, configured to judge whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed;

[0038] A positioning module is configured to extract feature information of the first stable object and the second stable object, and perform positioning based on a positioning mapping algorithm and the feature information.

[0039] In one embodiment, the classification module includes:

[0040] a first determining unit, configured to determine that the target object is a first stable object when a change in the position information of the target object is less than a first position change threshold;

[0041] The second determining unit is configured to determine that the target object is an object to be confirmed when the change in the position information of the target object is greater than or equal to the first position change threshold and less than a second position change threshold.

[0042] In one embodiment, the judgment module includes:

[0043] a detection unit, configured to perform object classification detection on the object to be confirmed in the feature image to obtain a category of the object to be confirmed;

[0044] a third determining unit, configured to determine that the object to be confirmed is a second stable object if the category of the object to be confirmed belongs to a preset category;

[0045] a sending unit, configured to generate an object confirmation request based on a characteristic image of the object to be confirmed and send the request to a target device if the category of the object to be confirmed does not belong to the preset category, wherein the object confirmation request is configured to prompt a user on the target device whether to select the object to be confirmed as a second stable object based on the characteristic image of the object to be confirmed;

[0046] The first receiving unit is configured to receive a selection result of a user and determine whether the object to be confirmed is a second stable object according to the selection result.

[0047] In one embodiment, the judgment module further includes:

[0048] an acquiring unit, configured to acquire prompt information, and send the characteristic image and the prompt information to a target device, wherein the prompt information is used to prompt a user on the target device whether to select the image to be confirmed as a second stable object based on the characteristic image of the object to be confirmed;

[0049] The second receiving unit is configured to receive a selection result of the user and determine whether the object to be confirmed is a second stable object according to the selection result.

[0050] In one embodiment, the first acquisition module includes:

[0051] a first acquiring unit, configured to respectively acquire first location information and second location information of the target object, wherein the first location information is location information of the target object at a first preset moment within the preset time period, and the second location information is location information of the target object at a second preset moment within the preset time period;

[0052] The fourth determining unit is configured to determine, based on the first location information and the second location information, a change in the location information of the target object within the preset time period.

[0053] In one embodiment, the apparatus further comprises:

[0054] a third acquisition module, configured to acquire, from the environmental image data, a first image feature of a preset position at a third preset moment and a second image feature of a fourth preset moment;

[0055] an identification module, configured to perform object identification on the first image feature and the second image feature respectively to obtain a first identified object and a second identified object;

[0056] a second determining module, configured to determine that a first stable object exists at the preset position when the first recognition object and the second recognition object are the same;

[0057] A third determining module is configured to determine that there is an object to be confirmed at the preset position when the first identified object and the second identified object are different.

[0058] In one embodiment, the apparatus further comprises:

[0059] A fourth acquisition module, configured to acquire a first image region corresponding to the first recognition object and a second image region corresponding to the second recognition object;

[0060] a calculation module, configured to calculate an intersection-over-union ratio of the first image area and the second image area;

[0061] The fourth determining module is configured to determine that the first identification object and the second identification object are the same when the IoU ratio satisfies a preset IoU ratio threshold, and otherwise determine that the first identification object and the second identification object are not the same.

[0062] In a third aspect, an embodiment of the present application provides a positioning device, including a memory and a processor;

[0063] The memory is used to store computer programs;

[0064] The processor is used to execute the computer program and implement the steps of the positioning method described in the first aspect when executing the computer program.

[0065] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the positioning method described in the first aspect above.

[0066] The embodiments of the present application provide a positioning method, apparatus, device, and storage medium. First, the method determines the amount of change in position information of each target object in the environmental image data within a preset time period; then, based on the amount of change in position information, each target object is classified to obtain a first stable object and an object to be confirmed, and then a feature image of the object to be confirmed is obtained; then, based on the feature image of the object to be confirmed, it is determined whether the object to be confirmed is a second stable object; and feature information of the first stable object and the second stable object is extracted, and positioning is performed based on the positioning mapping algorithm and the feature information. By classifying each target object based on the amount of change in position information of each target object in the environmental image data within a preset time period, determining the first stable object and the stable object to be confirmed in the environment, and further determining whether the stable object to be confirmed is the second stable object, it is possible to effectively prevent the missed detection of the second stable object, and then achieve positioning based on the feature information of the first stable object and the second stable object, which can effectively prevent positioning failure or drift problems, and aims to improve positioning accuracy.

[0067] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and do not limit the disclosure of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 A schematic diagram of the implementation flow of a positioning method provided in one embodiment of the present application;

[0070] Figure 2 A schematic diagram of an implementation flow of a positioning method provided in another embodiment of the present application;

[0071] Figure 3 A schematic diagram of an application scenario of the positioning method provided in an embodiment of the present application;

[0072] Figure 4 This is a schematic diagram of an environment image in an embodiment of the present application;

[0073] Figure 5 A schematic block diagram of a positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0075] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0076] It should be noted that the positioning method, apparatus, device and storage medium provided in the embodiments of the present application can effectively prevent positioning failure or drift problems and improve positioning accuracy.

[0077] See also Figure 1 As shown, Figure 1 The following is a flowchart illustrating an implementation of a positioning method according to an embodiment of the present application. The positioning method according to this embodiment is implemented by hardware or software of a positioning device.

[0078] It is understood that the positioning device is any device with image processing capabilities. Specifically, the positioning device can be a device with a central processing unit (CPU) and / or a graphics processing unit (GPU). The positioning device includes, but is not limited to, a terminal, a handheld smart device, or a robot. The positioning device can also have both a CPU and a GPU, and the terminal includes, but is not limited to, a personal computer or a workstation.

[0079] like Figure 1 As shown, the positioning method provided in the embodiment of the present application includes steps S101 to S106, which are described in detail as follows.

[0080] S101: Acquire environmental image data within a preset time period, and determine a change in position information of each target object in the environmental image data.

[0081] Among them, the positioning device can periodically obtain environmental image data at different times within a preset time period, or can obtain environmental image data at each time within the preset time period in real time. For example, environmental image data at different times within the preset time period can be obtained every 3 seconds, or environmental image data at each time within the preset time period can be obtained in real time. The environmental image data in this embodiment can be environmental image data of the geographical area where the positioning device is located, and can be directly obtained by a camera set on the positioning device, or can be obtained by a camera installed in the corresponding geographical area. Specifically, the environmental image includes a target object that can be located and navigated based on a positioning mapping algorithm. The positioning device can analyze the feature information of the target object in the environmental image based on the positioning mapping algorithm to achieve positioning.

[0082] In actual implementation, in order to improve the accuracy of positioning, it is necessary to use the characteristic information of the target object for positioning. However, since there may be moving objects in the geographical area, such as moving vehicles, pedestrians, or leaves swaying in the wind, etc., which are unstable objects, in order to avoid the influence of unstable objects on positioning accuracy, it is often necessary to extract stable objects from the environmental image data and perform navigation based on the characteristic information of the stable objects. However, since the shape and color of some fixed objects may also change rapidly, such as the outer wall of a building may fall off, or leaves may fall in the wind, etc., it may cause the stable objects to be missed during the navigation process, resulting in positioning failure or drift, and reducing the positioning accuracy. Therefore, in this embodiment, by analyzing the change in the position information of each target object in the acquired environmental image data, a first stable object and a second stable object are obtained from the environmental image respectively, to prevent the second stable object from being missed, effectively prevent the problem of positioning failure or drift, and improve positioning accuracy.

[0083] In one embodiment, determining the amount of change in position information of each target object in environmental image data includes: respectively obtaining first position information and second position information of each target object in the environmental image, wherein the first position information is the position information of the corresponding target object in the environmental image at a first preset moment within a preset time period, and the second position information is the position information of the corresponding target object at a second preset moment within the preset time period; based on the first position information and the second position information, respectively determining the amount of change in position information of each target object in the environmental image within the preset time period.

[0084] Specifically, each target object in the environmental image data and the first position information and second position information of each target object can be identified through a deep learning algorithm or a three-dimensional image recognition algorithm, and then based on the first position information and the second position information, the change in position information of each target object in the environmental image within a preset time period can be determined respectively.

[0085] For example, take the deep learning algorithm as an example. Specifically, it can be implemented by a pre-trained neural network model. Based on the pre-trained neural network model, semantic segmentation of the environmental image can be performed to segment the position of each target object in the pixel plane. After obtaining the position of each target object in the pixel plane, the positioning device further obtains the depth information of each target object, maps the depth information of each target object to the position of each target object in the pixel plane, and obtains the relative position information between the positioning device and each target object. The relative position information between the positioning device and each target object is then combined with the position information of the positioning device itself to calculate the position information of each target object. In this embodiment, the first position information and the second position information of each target object are both spatial position information.

[0086] S102 , classifying each target object according to a change in position information of each target object to obtain a first stable object and an object to be confirmed.

[0087] The change in the position information of the first stable object is less than the change in the position information of the object to be confirmed. Specifically, the first stable object is an object that does not move within a preset time period, or whose position information changes very little, close to not moving. The change in the position information of the object to be confirmed is greater than the change in the position information of the first stable object, but less than the change in the position information of a predetermined unstable object within a preset time period. Further classification of the object to be confirmed is required to determine whether it corresponds to the second stable target to prevent missed detection of stable objects and inaccurate positioning.

[0088] In one embodiment, each target object is classified according to the change in position information of each target object to obtain a first stable object and an object to be confirmed, including: when the change in position information of a target object is less than a first position change threshold, the target object is determined to be a first stable object; when the change in position information of a target object is greater than or equal to the first position change threshold and less than a second position change threshold, the target object is determined to be an object to be confirmed.

[0089] The first position change threshold is less than the second position change threshold, and the second position change threshold is less than a preset position information change of the unstable object within a preset time period. Specifically, the position information change of the unstable object within the preset time period can be predetermined based on experience and is not limited herein.

[0090] S103: Acquire a feature image of the object to be confirmed.

[0091] In a specific implementation, at least one of the color features, texture features, and shape features of the object to be confirmed in the environmental image can be extracted to obtain a feature image of the object to be confirmed. The color feature can characterize the surface properties of the object to be confirmed and is a pixel-based feature. Specifically, it can be obtained by obtaining the pixel values ​​of each pixel corresponding to the object to be confirmed in the environmental image. The texture feature can also characterize the surface properties of the object to be confirmed and is a feature obtained by statistical calculation based on multiple pixels. The shape feature can characterize the contour features or regional features of the object to be confirmed. The contour features are mainly for the outer boundary of the confirmed object, while the regional features are related to the shape area of ​​the object to be confirmed. Specifically, the contour features of the object to be confirmed can be identified through a neural network model, or the regional features of the object to be confirmed can be identified through a three-dimensional image recognition algorithm.

[0092] S104: Based on the characteristic image of the object to be confirmed, determine whether the object to be confirmed is a second stable object.

[0093] In one embodiment, based on the characteristic image of the object to be confirmed, determining whether the object to be confirmed is the second stable object includes: performing object classification detection on the characteristic image of the object to be confirmed to obtain the category of the object to be confirmed; if the category of the object to be confirmed belongs to a preset category, determining that the object to be confirmed is the second stable object; if the category of the object to be confirmed does not belong to the preset category, generating an object confirmation request based on the characteristic image of the object to be confirmed and sending it to a target device; wherein the object confirmation request is used to prompt a user on the target device whether to select the object to be confirmed as the second stable object based on the characteristic image of the object to be confirmed; receiving the user's selection result, and determining whether the object to be confirmed is the second object based on the user's selection result.

[0094] The second stable object may be predefined by the user. For example, the second object includes but is not limited to buildings, fixed objects inside or around buildings, fences, leaves of trees, mechanical devices with fixed positions but changeable postures, etc.

[0095] In specific implementations, the feature image of the object to be confirmed can be classified and detected based on a semantic segmentation algorithm to obtain the category of the object to be confirmed. The semantic segmentation algorithm can be implemented by a pre-trained semantic segmentation model; the pre-trained semantic segmentation model can assign an initial label category to each pixel of the feature image of the object to be confirmed based on a convolutional neural network (CNN). Its convolutional layer can effectively capture local features in the feature image, and many convolutional layers are nested together in a hierarchical manner, so that the CNN can extract more feature images. Through a series of convolutions, the CNN can capture the complex features of the feature image. The CNN can encode the content of the feature image of the object to be confirmed into a representation of the feature image and map it into an array of feature image category labels. This array of category labels can represent the category of the object to be confirmed. Of course, the feature image of the object to be confirmed can also be analyzed based on other preset classification algorithms to obtain the category of the object to be confirmed. Pre-set classification algorithms include, but are not limited to, Bayesian algorithms, random forest algorithms, k-classification algorithms, cloud clustering algorithms, etc., and are not limited to any of them.

[0096] In addition, there is an associated mapping relationship between the preset categories and the preset stability targets. For example, if the preset stability target is a building, the corresponding preset category is the building class; for another example, if the preset stability target is a staircase, the corresponding preset category is the building component, etc.

[0097] In one embodiment, based on the characteristic image of the object to be confirmed, determining whether the object to be confirmed is the second stable object includes: obtaining prompt information, and sending the characteristic image of the object to be confirmed and the prompt information to a target device, wherein the prompt information is used to prompt a user on the target device to select the second stable object in the characteristic image of the object to be confirmed; receiving the user's selection result, and determining whether the object to be confirmed is the second stable object based on the user's selection result.

[0098] In a specific implementation, the prompt information includes but is not limited to voice, text, characters, or any combination of text and characters. It should be understood that if the prompt information is text, characters, or a combination of text and characters, there is an associated mapping relationship between the prompt information and the second stable object.

[0099] The positioning device transmits the characteristic image and prompt information of the object to be confirmed to the target device. The target device may generate and display a corresponding target selection page, and display the characteristic image and prompt information of the object to be confirmed on the target selection page, prompting the user to select the second stable object from the characteristic image of the object to be confirmed based on the target selection interface. Correspondingly, the target selection interface displays a selection item for the second stable object, and the user may select the corresponding selection item by single-clicking, double-clicking, or touching to select the object to be confirmed as the second stable object.

[0100] In addition, after detecting the second stable object selected by the user, the target device generates a user selection result and sends the user selection result to the positioning device.

[0101] In this embodiment, to reduce the computational workload of the positioning device and improve its navigation efficiency, a characteristic image of the object to be confirmed is sent to a target device, and a prompt is provided to prompt the user on the target device to select whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed. The target device can be a user device communicatively connected to the positioning device, such as a smartwatch, smart remote control device, or handheld terminal.

[0102] It should be understood that when the category of the object to be confirmed is identified as the same as the preset category, or when a user selection is received from the target device indicating that the object to be confirmed is the second stable object, the object to be confirmed is determined to be the second stable object. This effectively prevents missed detection of stable objects and allows positioning based on the characteristic information of different stable objects, such as the first stable object and the second stable object, to not only effectively prevent positioning drift or failure but also improve positioning accuracy.

[0103] S105 , extracting feature information of the first stable object and the second stable object, and performing positioning based on a positioning and mapping algorithm and the feature information.

[0104] Among them, the positioning and mapping algorithm can be a VSLAM positioning algorithm; in the specific implementation, the VSLAM positioning algorithm performs positioning based on the feature information of the first stable object and the second stable object, which can effectively prevent positioning failure or drift caused by missed detection of stable objects, and improve positioning accuracy.

[0105] From the above analysis, it can be seen that the positioning method provided by the above embodiment classifies each target object according to the change in position information of each target object in the environmental image data within a preset time period, determines the first stable object and the stable object to be confirmed in the environment, and further determines whether the stable object to be confirmed is the second stable object. It can effectively prevent the missed detection of the second stable object, and then realize positioning according to the characteristic information of different stable objects, such as the first stable object and the second stable object, which can effectively prevent the problem of positioning failure or drift, and aims to improve the accuracy of positioning.

[0106] See also Figure 2 As shown, Figure 2 This is a schematic diagram of the implementation flow of the positioning method provided in another embodiment of the present application. Figure 1Compared with the illustrated embodiment, the specific implementation process of steps S206 to S207 is the same as that of steps S104 to S105, except that steps S101 to S103 are not included, and steps S201 to S205 are also included before step S206, as detailed below.

[0107] S201: Acquire environmental image data within a preset time period.

[0108] It should be understood that the process of obtaining environmental image data within a preset time period is similar to Figure 1 The process of obtaining environmental image data within a preset time period is the same in the embodiments and will not be repeated here.

[0109] S202 : Acquire, from the environmental image data of the preset time period, a first image feature of the preset location at a third preset moment and a second image feature at a fourth preset moment.

[0110] The preset position can be a position in the camera's field of view. There may be moving objects, such as moving vehicles, at this preset position. There may also be suspected moving objects, such as leaves moving in the wind. It should be understood that when leaves move in the wind, the feature points of the leaves acquired by the positioning device will also change. During the positioning process, the tree corresponding to the leaf may be missed as an unstable object, thereby causing positioning drift. To prevent missed detection of suspected moving objects, this embodiment acquires first and second image features of the preset position at different positioning times.

[0111] S203 , performing object recognition on the first image feature and the second image feature respectively to obtain a first recognized object and a second recognized object.

[0112] In a specific embodiment, object recognition can be performed on the first image feature and the second image feature based on a preset target object recognition algorithm. The preset target object recognition algorithm can be an object recognition algorithm based on deep learning or a three-dimensional depth recognition algorithm, and is not limited to any specific algorithm.

[0113] S204: When the first recognition object and the second recognition object are the same, determining that a first stable object exists at the preset position.

[0114] It should be understood that before step S204, a step of determining whether the first identification object and the second identification object are the same is also included. Exemplarily, the step of determining whether the first identification object and the second identification object are the same includes: obtaining a first image area corresponding to the first identification object and a second image area corresponding to the second identification object; calculating the intersection-and-union ratio of the first image area and the second image area; when the intersection-and-union ratio of the first image area and the second image area meets a preset intersection-and-union ratio threshold, determining that the first identification object and the second identification object are the same; otherwise, determining that the first identification object and the second identification object are not the same.

[0115] For example, the intersection-over-union ratio of the first image region and the second image region is recorded as IOU, and the IOU can be expressed as:

[0116]

[0117] Among them, area(C) represents the first image area, and area(G) represents the second image area.

[0118] The preset intersection-and-union ratio threshold is a pre-set minimum overlap ratio value for characterizing the first image area and the second image area. That is, when the intersection-and-union ratio of the first image area and the second image area is equal to the preset intersection-and-union ratio threshold, the overlap ratio characterizing the first image area and the second image area is greater than the minimum overlap ratio value, and the intersection-and-union ratio of the corresponding first image area and the second image area satisfies the preset intersection-and-union ratio threshold; otherwise, it is determined that the intersection-and-union ratio of the first image area and the second image area does not satisfy the preset intersection-and-union ratio threshold.

[0119] S205: When the first identified object and the second identified object are different, it is determined that there is an object to be confirmed at the preset position.

[0120] S206 : Determine whether the object to be confirmed is a second stable object based on the feature image of the object to be confirmed.

[0121] S207 , extracting feature information of the first stable object and the second stable object, and performing positioning based on a positioning and mapping algorithm and the feature information.

[0122] From the above analysis, it can be seen that the positioning method provided by the above embodiment, through object recognition of image features at the same preset position in the environmental image at different times, and after determining that there is an object to be confirmed at the preset position, further determines whether the object to be confirmed is a second stable object based on the feature image of the object to be confirmed, which can effectively prevent the missed detection of the second stable object, and then realize positioning according to the feature information of the target stable object, which can effectively prevent positioning failure or drift problems, and aims to improve the accuracy of positioning.

[0123] See also Figure 3 As shown, Figure 3A schematic diagram of an application scenario of the positioning method provided in the embodiment of the present application. Figure 3 It can be seen that the positioning method provided in this embodiment is applied to the navigation of the robot 320. It should be understood that the positioning method provided in this application includes but is not limited to applications such as Figure 3 In the application environment shown.

[0124] In this embodiment, the robot 320 is connected to the camera 340 via a network; the camera 340 may be a fixed camera set in the current environment, which can capture the current geographical area, such as a park, station, road, or community, to obtain an environmental image of the corresponding geographical area. The environmental image includes various objects such as houses, stairs, fences, and signboards; for example, Figure 4 As shown, the environmental image 420 includes not only the stable object house 4202 and the signboard 4204, but also the stable object tree 4206 to be confirmed. This is because the leaves of the tree 4206 will sway in the wind. When performing object category recognition, the tree 4206 is regarded as an object to be confirmed.

[0125] In addition, a positioning device 3200 is deployed in 320. The functions of the positioning device 3200 can be logically divided into multiple modules, each module can have different functions, and the functions of each module are realized by the processor in the robot 320 reading and executing instructions in the memory.

[0126] Exemplarily, the positioning device 3200 may include a first acquisition module 3201, a classification module 3202, a second acquisition module 3203, a judgment module 3204, a determination module 3205, and a positioning module 3206. In a specific implementation, the positioning device 3200 may execute the contents described in steps S101 to S106 described above, or the positioning device 3200 may execute the contents described in steps S201 to S208 described above. It should be noted that the present embodiment of the present application only provides an exemplary description of the structure and functional modules of the positioning device 3200.

[0127] The first acquisition module 3201 can acquire environmental image data within a preset time period from the camera 340 and determine the change in position information of each target object in the acquired environmental image data. The classification module 3202 can classify each target object in the environmental image data based on the change in position information of each target object determined by the first acquisition module 3201, thereby obtaining a first stable object and a to-be-confirmed object in the environmental image data. The second acquisition module 3203 can acquire a feature image of the to-be-confirmed object obtained by the classification module 3201. The judgment module 3204 can determine whether the to-be-confirmed object is a second stable object based on the feature image of the to-be-confirmed object obtained by the second acquisition module 3203. The positioning module 3205 can extract feature information of the first stable object and the second stable object and perform positioning based on the positioning mapping algorithm and the feature information. This allows positioning based on the feature information of the first stable object and the second stable object to effectively prevent positioning failure or drift and improve positioning accuracy.

[0128] Furthermore, in this embodiment, some of the multiple modules included in the robot 320 can also be merged into one module. For example, the above-mentioned first acquisition module 3201 and classification module 3202 can be merged into a determination module, that is, the determination module integrates the functions of the first acquisition module 3201 and the classification module 3202.

[0129] In an embodiment of the present application, a camera 340 may also be provided on the robot 320. As the robot 320 moves, the camera 340 provided on the robot 320 may capture images of the environment within its field of view. Specifically, the camera 340 may capture images of the environment within its field of view in real time, or may capture images of the environment within its field of view at preset time intervals. In addition, the positioning device 3200 may also be flexibly deployed on a terminal or server that is communicatively connected to the robot 320 to reduce the amount of computation required by the robot 320. For example, the positioning device 3200 may be deployed on a server 360 that is communicatively connected to the robot 320. The server 360 generates control instructions for controlling the movement of the robot by reading and executing instructions in the memory through a processor, and controls the robot to operate positioning and navigation according to the generated control instructions. The server may be a single server or a server cluster.

[0130] In addition, some functions of the positioning device 3200 can be deployed on the robot 320, and other functions can be deployed on the server 360 to achieve positioning and navigation of the robot 320. Specifically, the deployment can be flexibly performed according to the computing power of the robot 320, and no limitation is made here.

[0131] Combined with the above Figures 1 to 4, describes the positioning method provided by the present application in detail, and the positioning device and positioning equipment provided by the present application will be described below in conjunction with the accompanying drawings.

[0132] See also Figure 3 The system architecture diagram shown in FIG. 3 shows a schematic diagram of a positioning device, wherein the positioning device 3200 includes:

[0133] The first acquisition module 3201 is used to acquire environmental image data within a preset time period and determine the change in position information of each target object in the environmental image data;

[0134] A classification module 3202 is configured to classify the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed;

[0135] The second acquisition module 3203 is used to acquire the characteristic image of the object to be confirmed;

[0136] A judgment module 3204 is configured to judge whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed;

[0137] The positioning module 3205 is configured to extract feature information of the first stable object and the second stable object, and perform positioning based on a positioning mapping algorithm and the feature information.

[0138] In one embodiment, the classification module 3202 includes:

[0139] a first determining unit, configured to determine that the target object is a first stable object when a change in the position information of the target object is less than a first position change threshold;

[0140] The second determining unit is configured to determine that the target object is an object to be confirmed when the change in the position information of the target object is greater than or equal to the first position change threshold and less than a second position change threshold.

[0141] In one embodiment, the determination module 3204 includes:

[0142] a detection unit, configured to perform object classification detection on the feature image to obtain the category of the object to be confirmed;

[0143] a third determining unit, configured to determine that the object to be confirmed is the second stable object if the category of the object to be confirmed belongs to a preset category;

[0144] a sending unit, configured to generate an object confirmation request based on the feature image and send the request to a target device if the category of the object to be confirmed does not belong to the preset category, wherein the object confirmation request is configured to prompt a user on the target device whether to select the object to be confirmed as a second stable object based on the feature image of the object to be confirmed;

[0145] The first receiving unit is configured to receive a selection result of a user and determine whether the object to be confirmed is a second stable object according to the selection result.

[0146] In one embodiment, the determination module 3204 further includes:

[0147] an acquiring unit, configured to acquire prompt information, and send the feature image and the prompt information to a target device, wherein the prompt information is used to prompt a user on the target device to select a second stable object in the feature image;

[0148] The second receiving unit is configured to receive a selection result of the user and determine whether the object to be confirmed is a second stable object according to the selection result.

[0149] In one embodiment, the first acquisition module 3201 includes:

[0150] a first acquiring unit, configured to respectively acquire first location information and second location information of the target object, wherein the first location information is location information of the target object at a first preset moment within the preset time period, and the second location information is location information of the target object at a second preset moment within the preset time period;

[0151] The fourth determining unit is configured to determine, based on the first location information and the second location information, a change in the location information of the target object within the preset time period.

[0152] In one embodiment, the positioning device 3200 further includes:

[0153] a third acquisition module, configured to acquire, from the environmental image data, a first image feature of a preset position at a third preset moment and a second image feature of a fourth preset moment;

[0154] an identification module, configured to perform object identification on the first image feature and the second image feature respectively to obtain a first identified object and a second identified object;

[0155] a second determining module, configured to determine that a first stable object exists at the preset position when the first recognition object and the second recognition object are the same;

[0156] A third determining module is configured to determine that there is an object to be confirmed at the preset position when the first identified object and the second identified object are different.

[0157] In one embodiment, the positioning device 3200 further includes:

[0158] A fourth acquisition module, configured to acquire a first image region corresponding to the first recognition object and a second image region corresponding to the second recognition object;

[0159] a calculation module, configured to calculate an intersection-over-union ratio of the first image area and the second image area;

[0160] The fourth determining module is configured to determine that the first identification object and the second identification object are the same when the IoU ratio satisfies a preset IoU ratio threshold, and otherwise determine that the first identification object and the second identification object are not the same.

[0161] The object counting device according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module in the object counting device are respectively to implement Figure 1 or Figure 2 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0162] It should also be noted that the embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed across multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0163] See also Figure 5 As shown, Figure 5 A schematic block diagram of a positioning device provided in an embodiment of the present application.

[0164] like Figure 5 As shown, positioning device 500 includes a processor 501, memory 502, a communication interface 503, and a bus 504. The processor 501, memory 502, and communication interface 503 communicate via bus 504, and communication can also be achieved through other means such as wireless transmission. The memory 502 stores executable program code, and the processor 501 can call the program code stored in the memory 502 to execute the positioning method described in the aforementioned method embodiment.

[0165] It should be understood that in the embodiment of the present application, the processor 501 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0166] The memory 502 may include a read-only memory and a random access memory, and provides instructions and data to the processor 501. The memory 502 may also include a non-volatile random access memory. For example, the memory 502 may also store a data set.

[0167] The memory 502 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0168] In addition to the data bus, the bus 504 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as bus 504 in the figure.

[0169] It should be understood that the positioning device 500 according to the embodiment of the present application may correspond to the positioning apparatus in the embodiment of the present application, and may correspond to the apparatus for executing the positioning device in the embodiment of the present application. Figure 1 or Figure 2 The corresponding subjects in the method shown, and the above and other operations and / or functions of each device in the positioning device 500 are respectively to achieve Figure 1 or Figure 2 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0170] Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by means of dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, any function performed by a computer program can be easily implemented by the corresponding hardware. Moreover, the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits.

[0171] However, for this application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in each embodiment of this application.

[0172] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0173] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, they generate, in whole or in part, the processes or functions described in the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0174] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a training device or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A positioning method, characterized in that: The method comprises: Acquire environmental image data within a preset time period, and determine a change in position information of each target object in the environmental image data; Classifying the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed; Acquiring a characteristic image of the object to be confirmed; determining, based on the characteristic image of the object to be confirmed, whether the object to be confirmed is a second stable object; Extracting feature information of the first stable object and the second stable object, and performing positioning based on a positioning and mapping algorithm and the feature information; Wherein, judging whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed includes: Performing object classification detection on the object to be confirmed in the feature image to obtain a category of the object to be confirmed, wherein the feature image includes at least one of a color feature, a texture feature, and a shape feature of the object to be confirmed; If the category of the object to be confirmed belongs to the preset category, determining that the object to be confirmed is a second stable object; If the category of the object to be confirmed does not belong to the preset category, generating an object confirmation request based on the feature image of the object to be confirmed and sending it to the target device, wherein the object confirmation request is used to prompt a user on the target device whether to select the object to be confirmed as the second stable object based on the feature image of the object to be confirmed; A selection result of the user is received, and whether the object to be confirmed is a second stable object is determined according to the selection result.

2. The positioning method according to claim 1, wherein: The step of classifying the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed includes: When the change in the position information of the target object is less than a first position change threshold, determining that the target object is a first stable object; When the change in the position information of the target object is greater than or equal to the first position change threshold and less than the second position change threshold, the target object is determined to be an object to be confirmed.

3. The positioning method according to claim 1 or 2, characterized in that: The determining whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed further includes: Acquiring prompt information, and sending the characteristic image and prompt information to a target device, wherein the prompt information is used to prompt a user on the target device whether to select the object to be confirmed as a second stable object based on the characteristic image of the object to be confirmed; A selection result of the user is received, and whether the object to be confirmed is a second stable object is determined according to the selection result.

4. The positioning method according to claim 2, wherein: The determining of the change in position information of each target object in the environmental image data includes: Acquire first location information and second location information of the target object respectively, wherein the first location information is location information of the target object at a first preset moment within the preset time period, and the second location information is location information of the target object at a second preset moment within the preset time period; A change in the location information of the target object within the preset time period is determined based on the first location information and the second location information.

5. The positioning method according to claim 1, wherein: The method further comprises: acquiring, from the environmental image data, a first image feature of a preset position at a third preset moment and a second image feature of a fourth preset moment; Performing object recognition on the first image feature and the second image feature respectively to obtain a first recognized object and a second recognized object; When the first recognition object and the second recognition object are the same, determining that a first stable object exists at the preset position; When the first recognition object and the second recognition object are different, it is determined that there is an object to be confirmed at the preset position.

6. The positioning method according to claim 5, characterized in that: The method further comprises: Acquire a first image region corresponding to the first recognition object and a second image region corresponding to the second recognition object; calculating an intersection-over-union ratio of the first image area and the second image area; When the IoU meets a preset IoU threshold, it is determined that the first recognition object and the second recognition object are the same; otherwise, it is determined that the first recognition object and the second recognition object are not the same.

7. A positioning device, characterized in that: include: A first acquisition module is used to acquire environmental image data within a preset time period and determine a change in position information of each target object in the environmental image data; a classification module, configured to classify the target objects according to the change in the position information to obtain a first stable object and an object to be confirmed; A second acquisition module is used to acquire a characteristic image of the object to be confirmed; a judgment module, configured to judge whether the object to be confirmed is a second stable object based on the characteristic image of the object to be confirmed; The judgment module is specifically used for: Performing object classification detection on the object to be confirmed in the feature image to obtain a category of the object to be confirmed, wherein the feature image includes at least one of a color feature, a texture feature, and a shape feature of the object to be confirmed; If the category of the object to be confirmed belongs to the preset category, determining that the object to be confirmed is a second stable object; If the category of the object to be confirmed does not belong to the preset category, generating an object confirmation request based on the feature image of the object to be confirmed and sending it to the target device, wherein the object confirmation request is used to prompt a user on the target device whether to select the object to be confirmed as the second stable object based on the feature image of the object to be confirmed; receiving a selection result from a user, and determining whether the object to be confirmed is a second stable object according to the selection result; A positioning module is configured to extract feature information of the first stable object and the second stable object, and perform positioning based on a positioning mapping algorithm and the feature information.

8. A positioning device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the steps of the positioning method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the steps of the positioning method according to any one of claims 1 to 6.

Images