A high-precision positioning method integrating vision and AR technology

Abstract

The present invention discloses a high-precision positioning method integrating vision and AR technology, and relates to the field of intelligent positioning technology. The method comprises: semantically segmenting and labeling the captured image information of the target object, obtaining a target image labeled entity set, then traversing and extracting feature points of the target image labeled entity set based on an ORB detector, obtaining a target image entity feature point set, visually and virtually fusion the target image entity feature point set using AR technology, generating enhanced scene information of the target object, interactively obtaining the benchmark positioning information of the target image labeled entity set, spatially mapping the benchmark positioning information based on the enhanced scene information of the target object, and determining the target object position information for visual display. The method achieves the technical effect of realizing intelligent high-precision positioning by integrating vision and AR technology, reducing the influence of the accuracy of the use environment, improving the applicability of the positioning scene, and thus ensuring the precise and stable positioning.

Description

Technical Field

[0001] The present invention relates to the field of intelligent positioning technology, and in particular to a high-precision positioning method integrating vision and AR technologies. Background Art

[0002] With the advancement of science and technology, positioning technology plays an increasingly important role in various application scenarios such as industry, smart transportation, and shopping mall security. Among them, visual positioning is a positioning technology that determines the location of a device based on image coupling GPS data, using a series of images with known locations for positioning. However, traditional visual positioning methods often have problems such as low accuracy and being greatly affected by the environment. Summary of the invention

[0003] This application solves the technical problems of low visual positioning accuracy and great environmental impact in the prior art by providing a high-precision positioning method that integrates vision and AR technologies, thereby achieving the technical effect of realizing intelligent high-precision positioning by integrating vision and AR technologies, reducing the impact of the accuracy of the use environment, improving the applicability of positioning scenarios, and thereby ensuring the accuracy and stability of positioning.

[0004] In view of the above problems, the present invention provides a high-precision positioning method that integrates vision and AR technologies.

[0005] In the first aspect, the present application provides a high-precision positioning method that integrates vision and AR technologies, and the method includes: S1: capturing an image of a target object through a visual sensor to obtain captured image information of the target object, performing data preprocessing, semantic segmentation and labeling on the captured image information of the target object, and marking to obtain a target image labeled entity set; S2: initializing an ORB detector, and traversing feature points of each image entity in the target image labeled entity set in turn based on the ORB detector to obtain a target image entity feature point set; S3: using AR technology to describe the feature points of the target image entity feature point set, perform visual virtual fusion, and generate enhanced scene information of the target object; S4: interactively obtain the benchmark positioning information of the target image labeled entity set, perform spatial relationship mapping on the benchmark positioning information based on the target object enhanced scene information, and determine the target object position information for visual display.

[0006] On the other hand, the present application also provides a high-precision positioning system that integrates vision and AR technologies, and the system includes: an image labeling entity acquisition module, which is used to capture the image of the target object through a visual sensor, obtain the captured image information of the target object, perform data preprocessing and semantic segmentation labeling on the captured image information of the target object, and mark and obtain a target image labeling entity set; a traversal feature point extraction module, which is used to initialize the ORB detector, and perform traversal feature point extraction on each image entity in the target image labeling entity set in turn based on the ORB detector to obtain a target image entity feature point set; a visual virtual fusion module, which is used to use AR technology to describe the feature points of the target image entity feature point set, perform visual virtual fusion, and generate target object enhanced scene information; a position information determination module, which is used to interactively obtain the benchmark positioning information of the target image labeling entity set, perform spatial relationship mapping on the benchmark positioning information based on the target object enhanced scene information, and determine the target object position information for visual display.

[0007] In a third aspect, the present application provides an electronic device, comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, and the computer program, when executed by the processor, implements the steps of any one of the above methods.

[0008] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps in any one of the above-mentioned methods when executed by a processor.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] The method uses a visual sensor to capture the image of the target object, obtains the captured image information of the target object for data preprocessing, semantic segmentation and labeling, marks and obtains the target image labeled entity set, and then traverses the target image labeled entity set based on the ORB detector to extract feature points, obtains the target image entity feature point set, uses AR technology to perform visual virtual fusion on the target image entity feature point set, generates enhanced scene information of the target object, interactively obtains the reference positioning information of the target image labeled entity set, and then performs spatial relationship mapping on the reference positioning information based on the enhanced scene information of the target object, determines the target object position information for visual display. The technical effect of using fused vision and AR technology to achieve intelligent high-precision positioning, reduce the impact of the accuracy of the use environment, improve the applicability of positioning scenes, and ensure accurate and stable positioning is achieved.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flowchart of a high-precision positioning method integrating vision and AR technologies is provided for this application;

[0013] Figure 2 A schematic diagram of a process for obtaining a target image labeled entity set in a high-precision positioning method that integrates vision and AR technologies in this application;

[0014] Figure 3 A schematic diagram of the structure of a high-precision positioning system integrating vision and AR technology for this application;

[0015] Figure 4 This is a schematic diagram of the structure of an exemplary electronic device of the present application.

[0016] Explanation of the reference numerals: image labeling entity acquisition module 11, traversal feature point extraction module 12, visual virtual fusion module 13, position information determination module 14, bus 1110, processor 1120, transceiver 1130, bus interface 1140, memory 1150, operating system 1151, application 1152 and user interface 1160. DETAILED DESCRIPTION

[0017] In the description of the present application, those skilled in the art should know that the present application can be implemented as a method, an apparatus, an electronic device, and a computer-readable storage medium. Therefore, the present application can be specifically implemented in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), a combination of hardware and software. In addition, in some embodiments, the present application can also be implemented in the form of a computer program product in one or more computer-readable storage media, and the computer-readable storage medium contains computer program code.

[0018] The above-mentioned computer-readable storage medium may adopt any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, flash memories, optical fibers, optical disc read-only memories, optical storage devices, magnetic storage devices, or any combination of the above. In the present application, computer-readable storage media can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0019] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws.

[0020] The present application describes the provided methods, devices, and electronic devices through flowcharts and / or block diagrams.

[0021] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer-readable program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified by the boxes in the flowchart and / or block diagram.

[0022] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to work in a specific manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction device product including functions / operations specified in the blocks in the flowchart and / or block diagram.

[0023] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable data processing apparatus to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0024] The present application is described below in conjunction with the drawings in the present application.

[0025] Embodiment 1

[0026] like Figure 1As shown, the present application provides a high-precision positioning method integrating vision and AR technology, the method comprising:

[0027] S1: Capturing an image of a target object through a visual sensor to obtain captured image information of the target object, performing data preprocessing, semantic segmentation and labeling on the captured image information of the target object, and marking and obtaining a labeled entity set of the target image;

[0028] like Figure 2 As shown, further, the target image labeled entity set is obtained by marking in S1, and the steps of this application also include:

[0029] S11: performing noise identification and wavelet denoising on the captured image information of the target object by Fourier transform to obtain standard target captured image information;

[0030] S12: creating an image data dimensionality reduction rule, and mapping and reducing the dimensionality of the standard target captured image information based on the image data dimensionality reduction rule to obtain the target execution captured image information;

[0031] S13: constructing an entity tag library according to the application scenario information of the target object, and training an entity semantic segmentation network based on the entity tag library;

[0032] S14: Based on the entity semantic segmentation network, semantic segmentation and entity labeling are performed on the captured image information of the target to obtain the target image labeled entity set.

[0033] Furthermore, the step of acquiring the standard target captured image information in S11, the present application also includes:

[0034] Using Fourier transform to perform frequency domain conversion and noise characteristic extraction on the captured image information of the target object to obtain image noise characteristic information, and determining the image signal wavelet selection threshold according to the image noise characteristic information;

[0035] Performing wavelet decomposition on the image signal of the target object capturing the image information to obtain the target image signal wavelet coefficients;

[0036] According to the image signal wavelet selection threshold, the wavelet coefficients of the target image signal are intercepted, the noise signal smaller than the image signal wavelet selection threshold is set to zero, and the available image signal information larger than the image signal wavelet selection threshold is obtained;

[0037] The available image signal information is filtered and reconstructed to obtain the standard target captured image information.

[0038] Furthermore, the target execution captured image information is obtained in S12, and the steps of the present application also include:

[0039] Performing object distance identification on the standard target captured image information based on the target object to determine image target distance level information;

[0040] Obtaining the definition of the target image, and performing dimensionality reduction analysis on the image target distance level information and the target image definition based on the image data dimensionality reduction rule to determine the pixel data density information;

[0041] The standard target captured image information is enhanced based on the pixel data density information to obtain the target execution captured image information.

[0042] Specifically, in order to achieve intelligent high-precision positioning, the image of the target object is captured by a visual sensor, wherein the visual sensor can be a camera, an image acquisition device, etc., for capturing the image of the target object, and the target object is the object to be positioned, which can be a user, an industrial robot, etc., so as to obtain the captured image information of the target object collected at the current object position. The captured image information of the target object is subjected to data preprocessing, semantic segmentation and labeling for subsequent image augmented reality processing. Specifically, the captured image information of the target object is subjected to noise identification and wavelet denoising by using Fourier transform. First, the captured image information of the target object is converted into the frequency domain by the Fourier transform algorithm, and it is converted from the spatial domain to the frequency domain. Then, the noise characteristics of the frequency domain image are extracted. Noise belongs to the high-frequency component in the spectrum diagram. The image noise characteristic information is obtained by comparison and identification. The image noise characteristic information includes noise intensity, distribution, etc.

[0043] The image signal wavelet selection threshold is determined according to the image noise characteristic information. The image signal wavelet selection threshold is the spectral coefficient selection range of the image effective signal. After wavelet decomposition, the wavelet coefficient amplitude of the image signal is greater than the wavelet coefficient amplitude of the image noise. The image signal of the target object capturing image information is subjected to wavelet decomposition and extraction by a wavelet basis extraction function. Common wavelet basis functions include Haar wavelet, Morlet wavelet, etc., and the target image signal wavelet coefficient is extracted. The target image signal wavelet coefficient is the projection of the wavelet function space selected when performing wavelet decomposition. The image signal is decomposed into various scales by wavelet decomposition. The target image signal wavelet coefficient is intercepted according to the image signal wavelet selection threshold, and the noise signal less than the image signal wavelet selection threshold is set to zero, that is, the wavelet coefficient belonging to the noise at each scale is removed, and the available image signal information greater than the image signal wavelet selection threshold is obtained, that is, the wavelet coefficient belonging to the signal is retained and enhanced. The available image signal information is filtered and reconstructed, that is, the processed wavelet coefficient is reconstructed by wavelet inverse transform to obtain the standard target captured image information after filtering and denoising. The noise of the target image is suppressed while retaining the image details, thereby improving the quality of subsequent image processing and application reliability.

[0044] Since the amount of pixel data of the collected image is large, the image data needs to be processed by dimensionality reduction, so an image data dimensionality reduction rule is created. The image data dimensionality reduction rule is the basis for the dimensionality reduction processing of the image pixel data, and the pixel data is reduced by the pixel demand density. Based on the image data dimensionality reduction rule, the standard target captured image information is mapped and reduced in dimension. First, the standard target captured image information is marked with an object distance based on the target object, that is, the pixel distances between other areas in the image and the target object are marked with the target object as the center, so as to determine the image target distance level information between each image area and the target object. The target image clarity is set empirically, and the target image clarity is the lowest pixel clarity of each image area. Then, based on the image data dimensionality reduction rule, the image target distance level information is mapped and reduced in dimension analysis with the target image clarity, that is, the pixel density of each image area is uniformly decreased according to the distance level. The closer the image area is to the target object, the greater the pixel density is. In this way, the distance level mapping and dimensionality reduction are performed to determine the pixel data density information of each image area, and ensure that the pixel density of each image area after dimensionality reduction meets the preset target image clarity. The standard target captured image information is subjected to dimensionality reduction and enhancement based on the pixel data density information to obtain the target executed captured image information after dimensionality reduction processing.

[0045] An entity label library is constructed according to the application scenario information of the target object. The entity label library is the type of elements that appear in the image. For example, for the office building application scenario, its entity labels include walls, office chairs, corridors, printers, etc. Based on the entity label library, image region segmentation training is performed on the collected scene image data set, and the same image regions are marked as the same type of labels to obtain an entity semantic segmentation network. The entity semantic segmentation network is used to perform regional labeling and rapid segmentation of the target image. Based on the entity semantic segmentation network, semantic segmentation is performed on the captured image information of the target, and the image region is segmented according to the entity label, and then the segmented entity region is entity labeled to obtain a target image labeled entity set. The target image labeled entity set is the image entity element information after semantic segmentation processing, such as office chairs, corridors, printers, etc. Entity labeling segmentation processing of visual images is realized, thereby improving the quality of subsequent visual fusion enhancement.

[0046] Step S2: Initialize the ORB detector, and extract feature points of each image entity in the target image labeled entity set in turn based on the ORB detector to obtain a target image entity feature point set;

[0047] Specifically, the ORB algorithm is used to extract feature points from the image. First, the ORB detector is initialized. Its initialization parameters include the number of pyramid layers, which determines the scale invariance of feature point detection; the edge threshold, which is used in the FAST corner detection algorithm; and the maximum number of feature points, etc. Based on the ORB detector, feature points of each image entity in the target image labeled entity set are traversed and extracted in turn to obtain a target image entity feature point set. The target image entity feature point set is the key feature point of each image entity, which has strong grayscale changes, texture information or local structure, including corner points, edge points, etc., to improve the accuracy of subsequent image feature point matching and visual virtual fusion.

[0048] Step S3: using AR technology to describe the target image entity feature point set, perform visual virtual fusion, and generate enhanced scene information of the target object;

[0049] Furthermore, the target object enhanced scene information is generated in S3, and the steps of this application also include:

[0050] S31: Obtain image feature description factors, where the image feature description factors include color, texture, edge, gradient, and spatial relationship;

[0051] S32: performing feature point description on the target image entity feature point set based on the image feature description factor, and determining an image entity feature point description subset;

[0052] S33: performing registration and fusion on the target image entity feature point set based on the image entity feature point descriptor set to obtain an image entity registration feature point set;

[0053] S34: The image entity registration feature point set is visually aligned and fused, and rendered in real time through AR technology to generate enhanced scene information of the target object.

[0054] Furthermore, in the step S33, a set of image entity registration feature points is obtained, and the steps of the present application further include:

[0055] Performing similarity analysis on each feature descriptor in the image entity feature point descriptor set to obtain an image entity feature point similarity set;

[0056] Obtaining a preset similarity threshold, and performing registration screening on the image entity feature point similarity set based on the preset similarity threshold to obtain a registration feature point similarity set greater than the preset similarity threshold;

[0057] The target image entity feature point set is registered and fused based on the registration feature point similarity set to determine the image entity registration feature point set.

[0058] Specifically, the target image entity feature point set is subjected to feature point description and visual virtual fusion by using AR technology. First, image feature description factors are obtained. The image feature description factors are description dimensions of image feature points, including color, texture, edge, gradient, spatial relationship, etc. Then, multi-dimensional feature description is performed on each entity feature point in the target image entity feature point set based on the image feature description factors, and the corresponding image entity feature point description subset is determined. The target image entity feature point set is registered and fused based on the image entity feature point description subset, specifically, a similarity algorithm is used to perform similarity analysis on each feature descriptor in the image entity feature point description subset, and the image entity feature point similarity set between each entity feature point is calculated. A preset similarity threshold is obtained empirically, and the preset similarity threshold is the feature point matching similarity. The image entity feature point similarity set is registered and screened based on the preset similarity threshold to obtain a registration feature point similarity set greater than the preset similarity threshold.

[0059] Based on the similarity set of the registration feature points, the target image entity feature point set is reversely registered and fused to determine the matched image entity registration feature point set. The image entity registration feature point set is visually aligned and fused through AR technology, and the position and posture of the virtual image entity object are aligned and simulated with the feature points in the real world. Then, the fused image is rendered to the user in real time through image rendering technologies such as texture mapping and lighting models, and target object enhanced scene information is generated to display the augmented reality scene in real time on the user's AR device, such as AR glasses, smart phones, etc., so that the user can see the virtual object position mark information in the real environment. Intelligent high-precision positioning is achieved by using fusion vision and AR technology, reducing the impact of the accuracy of the use environment, improving the applicability of positioning scenes, and thus ensuring accurate and stable positioning.

[0060] Step S4: interactively obtain the reference positioning information of the target image labeled entity set, perform spatial relationship mapping on the reference positioning information based on the target object enhanced scene information, and determine the target object position information for visual display.

[0061] Furthermore, the application steps also include:

[0062] Configuring a positioning inertial measurement sensor for the target object, monitoring the motion state of the target object based on the positioning inertial measurement sensor, and acquiring target motion state parameter information;

[0063] Constructing an object motion prediction model, and performing motion variable prediction on the target motion state parameter information based on the object motion prediction model to obtain target motion prediction variable information;

[0064] A positioning error factor is determined according to the target motion prediction variable information, and the target object position information is dynamically optimized based on the positioning error factor to obtain the target object dynamic position information.

[0065] Specifically, the reference positioning information of the target image labeled entity set is obtained by interacting with the GPS map, and the reference positioning information is the map position coordinates of each scene entity. The position relationship between the target object and other scene entities in three-dimensional space, such as distance, angle, etc., is established through the target object enhanced scene information, and then the reference positioning information is spatially mapped based on the target object enhanced scene information, and the target object position information in reality is determined and visualized. The display device can be a mobile phone, a tablet computer, a display, etc., which is used to display the precise position information of the target object for subsequent indication and control of the target object.

[0066] Under long-term or high-speed motion, the positioning accuracy of the target object will be affected. Therefore, a positioning inertial measurement sensor is configured for the target object. The positioning inertial measurement sensor is a positioning device that integrates sensors such as accelerometers and gyroscopes. Based on the positioning inertial measurement sensor, the motion state of the target object is monitored to obtain target motion state parameter information, which includes speed, acceleration, angular velocity, etc. The object motion prediction model is constructed by fitting the object motion data. The object motion prediction model is used to predict the object motion change parameters. The target motion state parameter information is predicted based on the object motion prediction model. The target motion prediction variable information is output to obtain the target motion prediction variable information. The target motion prediction variable information is the predicted motion distance and direction deviation parameters of the target object in the current motion state. The positioning error factor is determined according to the target motion prediction variable information, and the target object position information is dynamically optimized based on the positioning error factor to obtain the target object dynamic position information. The target object dynamic position information is the positioning information of the target object in motion. Considering the motion deviation influencing parameters, the motion positioning error is reduced, thereby improving the actual positioning accuracy.

[0067] In summary, the high-precision positioning method provided by this application that integrates vision and AR technology has the following technical effects:

[0068] The method uses a visual sensor to capture the image of the target object, obtains the captured image information of the target object for data preprocessing, semantic segmentation and labeling, marks and obtains the target image labeled entity set, and then traverses the target image labeled entity set based on the ORB detector to extract feature points, obtains the target image entity feature point set, uses AR technology to perform visual virtual fusion on the target image entity feature point set, generates enhanced scene information of the target object, interactively obtains the reference positioning information of the target image labeled entity set, and then performs spatial relationship mapping on the reference positioning information based on the enhanced scene information of the target object, determines the target object position information for visual display. The technical effect of using fused vision and AR technology to achieve intelligent high-precision positioning, reduce the impact of the accuracy of the use environment, improve the applicability of positioning scenes, and ensure accurate and stable positioning is achieved.

[0069] Embodiment 2

[0070] Based on the same inventive concept as the high-precision positioning method integrating vision and AR technology in the aforementioned embodiment, the present invention also provides a high-precision positioning system integrating vision and AR technology, such as Figure 3 As shown, the system comprises:

[0071] The image labeling entity acquisition module 11 is used to capture an image of a target object through a visual sensor, obtain captured image information of the target object, perform data preprocessing and semantic segmentation labeling on the captured image information of the target object, and mark and obtain a target image labeling entity set;

[0072] A traversal feature point extraction module 12 is used to initialize an ORB detector, and perform traversal feature point extraction on each image entity in the target image labeled entity set in turn based on the ORB detector to obtain a target image entity feature point set;

[0073] A visual virtual fusion module 13 is used to use AR technology to describe the feature points of the target image entity feature point set, perform visual virtual fusion, and generate enhanced scene information of the target object;

[0074] The position information determination module 14 is used to interactively obtain the reference positioning information of the target image labeled entity set, perform spatial relationship mapping on the reference positioning information based on the target object enhanced scene information, and determine the target object position information for visual display.

[0075] Furthermore, the system also includes:

[0076] A noise identification and denoising unit, used to perform noise identification and wavelet denoising on the captured image information of the target object by Fourier transform, so as to obtain the captured image information of the standard target;

[0077] An image dimension reduction unit, used for creating an image data dimension reduction rule, and mapping and reducing the dimension of the standard target captured image information based on the image data dimension reduction rule to obtain the target execution captured image information;

[0078] A semantic segmentation network acquisition unit, configured to construct an entity label library according to application scenario information of the target object, and obtain an entity semantic segmentation network based on training of the entity label library;

[0079] An entity labeling unit is used to perform semantic segmentation and entity labeling on the captured image information of the target based on the entity semantic segmentation network, and obtain the target image labeled entity set.

[0080] Furthermore, the system also includes:

[0081] A wavelet selection threshold determination unit is used to perform frequency domain conversion and noise characteristic extraction on the captured image information of the target object by Fourier transform to obtain image noise characteristic information, and determine the image signal wavelet selection threshold according to the image noise characteristic information;

[0082] A wavelet decomposition unit, used for performing wavelet decomposition on the image signal of the target object capturing the image information, to obtain the wavelet coefficients of the target image signal;

[0083] A usable image signal acquisition unit, used to intercept the target image signal wavelet coefficients according to the image signal wavelet selection threshold, set the noise signal smaller than the image signal wavelet selection threshold to zero, and acquire the usable image signal information larger than the image signal wavelet selection threshold;

[0084] The filtering and reconstruction unit is used to filter and reconstruct the available image signal information to obtain the standard target captured image information.

[0085] Furthermore, the system also includes:

[0086] An object distance identification unit, configured to perform object distance identification on the standard target captured image information based on the target object, and determine image target distance level information;

[0087] A mapping dimension reduction analysis unit is used to obtain the definition of the target image, and to perform a mapping dimension reduction analysis on the image target distance level information and the definition of the target image based on the image data dimension reduction rule to determine the pixel data density information;

[0088] An image information enhancement unit is used to enhance the standard target captured image information based on the pixel data density information to obtain the target execution captured image information.

[0089] Furthermore, the system also includes:

[0090] A feature description factor acquisition unit, used to acquire image feature description factors, wherein the image feature description factors include color, texture, edge, gradient, and spatial relationship;

[0091] A feature point description unit, used to perform feature point description on the target image entity feature point set based on the image feature description factor, and determine an image entity feature point description subset;

[0092] A registration feature point acquisition unit, used for performing registration fusion on the target image entity feature point set based on the image entity feature point description subset set to obtain an image entity registration feature point set;

[0093] The real-time rendering unit is used to perform visual alignment fusion and real-time rendering on the image entity registration feature point set through AR technology to generate enhanced scene information of the target object.

[0094] Furthermore, the system also includes:

[0095] A similarity analysis unit, used for performing similarity analysis on each feature descriptor in the image entity feature point descriptor set to obtain an image entity feature point similarity set;

[0096] A registration screening unit, used to obtain a preset similarity threshold, and perform registration screening on the image entity feature point similarity set based on the preset similarity threshold to obtain a registration feature point similarity set greater than the preset similarity threshold;

[0097] The feature registration and fusion unit is used to register and fuse the target image entity feature point set based on the registration feature point similarity set to determine the image entity registration feature point set.

[0098] Furthermore, the system also includes:

[0099] A motion state monitoring unit, configured to configure a positioning inertial measurement sensor for the target object, monitor the motion state of the target object based on the positioning inertial measurement sensor, and obtain target motion state parameter information;

[0100] A motion variable prediction unit, used for constructing an object motion prediction model, and performing motion variable prediction on the target motion state parameter information based on the object motion prediction model to obtain target motion prediction variable information;

[0101] The position dynamic optimization unit is used to determine the positioning error factor according to the target motion prediction variable information, and dynamically optimize the target object position information based on the positioning error factor to obtain the target object dynamic position information.

[0102] The foregoing Figure 1 The various variations and specific examples of a high-precision positioning method that integrates vision and AR technology in Example 1 are also applicable to a high-precision positioning system that integrates vision and AR technology in this embodiment. Through the above detailed description of a high-precision positioning method that integrates vision and AR technology, those skilled in the art can clearly know the implementation method of a high-precision positioning system that integrates vision and AR technology in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0103] In addition, the present application also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and run on the processor. The transceiver, the memory, and the processor are respectively connected via a bus. When the computer program is executed by the processor, each process of the above-mentioned method embodiment for controlling output data is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0104] Exemplary Electronic Devices

[0105] For details, see Figure 4 As shown, the present application also provides an electronic device, which includes a bus 1110 , a processor 1120 , a transceiver 1130 , a bus interface 1140 , a memory 1150 and a user interface 1160 .

[0106] In the present application, the electronic device further includes: a computer program stored in the memory 1150 and executable on the processor 1120, and when the computer program is executed by the processor 1120, each process of the above-mentioned method embodiment for controlling output data is implemented.

[0107] The transceiver 1130 is configured to receive and send data under the control of the processor 1120 .

[0108] In the present application, a bus architecture (represented by bus 1110 ) may include any number of interconnected buses and bridges, and bus 1110 connects various circuits including one or more processors represented by processor 1120 and a memory represented by memory 1150 .

[0109] Bus 1110 represents one or more of any of several types of bus structures, including a memory bus and memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. Such architectures include, by way of example and not limitation, an Industry Standard Architecture bus, a Micro Channel Architecture bus, an expansion bus, a Video Electronics Standards Association, a Peripheral Component Interconnect bus.

[0110] The processor 1120 may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method embodiment may be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processors include: a general-purpose processor, a central processing unit, a network processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, a complex programmable logic device, a programmable logic array, a microcontroller unit or other programmable logic device, a discrete gate, a transistor logic device, and a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in this application may be implemented or executed. For example, the processor may be a single-core processor or a multi-core processor, and the processor may be integrated into a single chip or located on multiple different chips.

[0111] Processor 1120 may be a microprocessor or any conventional processor. The method steps disclosed in the present application may be directly executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a readable storage medium known in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, a register, etc. The readable storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0112] The bus 1110 may also connect various other circuits such as peripheral devices, voltage regulators or power management circuits, and the bus interface 1140 provides an interface between the bus 1110 and the transceiver 1130, which are well known in the art. Therefore, this application will not further describe them.

[0113] The transceiver 1130 may be one element or multiple elements, such as multiple receivers and transmitters, and provides a unit for communicating with various other devices on a transmission medium. For example, the transceiver 1130 receives external data from other devices, and the transceiver 1130 is used to send data processed by the processor 1120 to other devices. Depending on the nature of the computer device, a user interface 1160 may also be provided, such as a touch screen, a physical keyboard, a display, a mouse, a speaker, a microphone, a trackball, a joystick, or a stylus.

[0114] It should be understood that in the present application, the memory 1150 may further include a memory remotely arranged relative to the processor 1120, and these remotely arranged memories may be connected to the server through a network. One or more parts of the above network may be a self-organizing network, an intranet, an extranet, a virtual private network, a local area network, a wireless local area network, a wide area network, a wireless wide area network, a metropolitan area network, the Internet, a public switched telephone network, a plain old telephone service network, a cellular telephone network, a wireless network, a wireless fidelity network, and a combination of two or more of the above networks. For example, the cellular telephone network and the wireless network may be a global mobile communication device, a code division multiple access device, a global microwave interconnection access device, a general packet radio service device, a wideband code division multiple access device, a long term evolution device, a LTE frequency division duplex device, a LTE time division duplex device, an advanced long term evolution device, a universal mobile communication device, an enhanced mobile broadband device, a massive machine type communication device, an ultra-reliable low latency communication device, and the like.

[0115] It should be understood that the memory 1150 in the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory includes: a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or a flash memory.

[0116] Volatile memory includes random access memory, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory, dynamic random access memory, synchronous dynamic random access memory, double data rate synchronous dynamic random access memory, enhanced synchronous dynamic random access memory, synchronously linked dynamic random access memory, and direct memory bus random access memory. The memory 1150 of the electronic device described in the present application includes, but is not limited to, the above and any other suitable types of memory.

[0117] In the present application, the memory 1150 stores the following elements of the operating system 1151 and the application program 1152: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0118] Specifically, the operating system 1151 includes various device programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 1152 includes various application programs, such as a media player and a browser, which are used to implement various application services. The program implementing the method of the present application may be included in the application 1152. The application 1152 includes applets, objects, components, logic, data structures, and other computer device executable instructions that perform specific tasks or implement specific abstract data types.

[0119] In addition, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned method embodiment for controlling output data are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0120] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A high-precision positioning method integrating vision and AR technology, characterized in that: The method comprises: S1: Capturing an image of a target object through a visual sensor to obtain captured image information of the target object, performing data preprocessing, semantic segmentation and labeling on the captured image information of the target object, and marking and obtaining a labeled entity set of the target image; S2: Initialize an ORB detector, and extract feature points of each image entity in the target image labeled entity set in turn based on the ORB detector to obtain a target image entity feature point set; S3: Using AR technology to describe the feature points of the target image entity set, perform visual virtual fusion, and generate enhanced scene information of the target object; S4: interactively acquiring reference positioning information of the target image labeled entity set, performing spatial relationship mapping on the reference positioning information based on the target object enhanced scene information, and determining the target object position information for visual display; The target object enhanced scene information is generated in S3, including: S31: Obtain image feature description factors, where the image feature description factors include color, texture, edge, gradient, and spatial relationship; S32: performing feature point description on the target image entity feature point set based on the image feature description factor, and determining an image entity feature point description subset; S33: performing registration and fusion on the target image entity feature point set based on the image entity feature point descriptor set to obtain an image entity registration feature point set; S34: The image entity registration feature point set is visually aligned and fused, and rendered in real time through AR technology to generate enhanced scene information of the target object.

2. The method according to claim 1, characterized in that The target image labeled entity set is obtained by marking in S1, including: S11: performing noise identification and wavelet denoising on the captured image information of the target object by Fourier transform to obtain standard target captured image information; S12: creating an image data dimensionality reduction rule, and mapping and reducing the dimensionality of the standard target captured image information based on the image data dimensionality reduction rule to obtain the target execution captured image information; S13: constructing an entity label library according to the application scenario information of the target object, and training an entity semantic segmentation network based on the entity label library; S14: Based on the entity semantic segmentation network, semantic segmentation and entity labeling are performed on the captured image information of the target to obtain the target image labeled entity set.

3. The method according to claim 2, characterized in that The step S11 of acquiring standard target captured image information includes: Using Fourier transform to perform frequency domain conversion and noise characteristic extraction on the captured image information of the target object to obtain image noise characteristic information, and determining the image signal wavelet selection threshold according to the image noise characteristic information; Performing wavelet decomposition on the image signal of the target object capturing the image information to obtain the target image signal wavelet coefficients; According to the image signal wavelet selection threshold, the wavelet coefficients of the target image signal are intercepted, the noise signal smaller than the image signal wavelet selection threshold is set to zero, and the available image signal information larger than the image signal wavelet selection threshold is obtained; The available image signal information is filtered and reconstructed to obtain the standard target captured image information.

4. The method according to claim 2, characterized in that The step S12 obtains the target execution captured image information, including: Performing object distance identification on the standard target captured image information based on the target object to determine image target distance level information; Obtaining the definition of the target image, and performing dimensionality reduction analysis on the image target distance level information and the target image definition based on the image data dimensionality reduction rule to determine the pixel data density information; The standard target captured image information is enhanced based on the pixel data density information to obtain the target execution captured image information.

5. The method according to claim 1, characterized in that The image entity registration feature point set is obtained in S33, including: Performing similarity analysis on each feature descriptor in the image entity feature point descriptor set to obtain an image entity feature point similarity set; Obtaining a preset similarity threshold, and performing registration screening on the image entity feature point similarity set based on the preset similarity threshold to obtain a registration feature point similarity set greater than the preset similarity threshold; The target image entity feature point set is registered and fused based on the registration feature point similarity set to determine the image entity registration feature point set.

6. The method according to claim 1, characterized in that The method comprises: Configuring a positioning inertial measurement sensor for the target object, monitoring the motion state of the target object based on the positioning inertial measurement sensor, and acquiring target motion state parameter information; Constructing an object motion prediction model, and performing motion variable prediction on the target motion state parameter information based on the object motion prediction model to obtain target motion prediction variable information; A positioning error factor is determined according to the target motion prediction variable information, and the target object position information is dynamically optimized based on the positioning error factor to obtain the target object dynamic position information.

7. A high-precision positioning system integrating vision and AR technology, characterized in that: A high-precision positioning method integrating vision and AR technology for implementing any one of claims 1 to 6, the system comprising: An image labeling entity acquisition module is used to capture an image of a target object through a visual sensor, obtain captured image information of the target object, perform data preprocessing and semantic segmentation labeling on the captured image information of the target object, and mark and obtain a target image labeling entity set; A traversal feature point extraction module is used to initialize the ORB detector, and perform traversal feature point extraction on each image entity in the target image labeled entity set in turn based on the ORB detector to obtain a target image entity feature point set; A visual virtual fusion module is used to use AR technology to describe the feature points of the target image entity set, perform visual virtual fusion, and generate enhanced scene information of the target object; The position information determination module is used to interactively obtain the reference positioning information of the target image labeled entity set, perform spatial relationship mapping on the reference positioning information based on the target object enhanced scene information, and determine the target object position information for visual display.

8. An electronic device, comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, wherein: When the computer program is executed by the processor, the steps in the high-precision positioning method integrating vision and AR technology as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in a high-precision positioning method integrating vision and AR technology as described in any one of claims 1 to 6 are implemented.

Images