Indoor positioning method, device and electronic equipment

By inputting the image frames of two-dimensional information into the trained indoor positioning model, using feature extraction and image generation networks, efficient and accurate three-dimensional positioning of indoor equipment is achieved, and the indoor positioning time and low reliability in the prior art are solved. The problem.

CN114353796BActive Publication Date: 2024-11-08COAL SCI (BEIJING) TESTING TECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202111511699.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-06
Publication Date
2024-11-08
Estimated Expiration
2041-12-06

AI Technical Summary

Technical Problem

The prior art lacks an indoor positioning method that consumes short time, is highly efficient and has high reliability, and it is difficult to accurately locate indoor equipment.

Method used

By acquiring the first image frame of the two-dimensional information captured by the image acquisition device, input it into the target indoor positioning model obtained based on three-dimensional data training, the three-dimensional positioning results are output, and feature extraction and image processing are used to use the feature extraction network and the image generation network. Obtain the three-dimensional positioning results and movement trajectory of the device.

Benefits of technology

It shortens the time-consuming indoor positioning, improves the accuracy and reliability of the positioning process, avoids complex operations, and realizes real-time three-dimensional positioning results acquisition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114353796B_ABST
    Figure CN114353796B_ABST
Patent Text Reader

Abstract

The present application provides an indoor positioning method, device and electronic device, the method comprising: obtaining a first image frame captured by an image acquisition device, wherein the image acquisition device is arranged on any component of any device in a target area; inputting the first image frame into a trained target indoor positioning model, and outputting a first three-dimensional positioning result of any component corresponding to the first image frame. In the process of indoor positioning, the present application can only rely on the first image frame with two-dimensional information, and process it through the target indoor positioning model trained based on three-dimensional data such as indoor device appearance data and indoor environment data. Optionally, relative position relationship analysis is performed based on the first image frame to obtain the first three-dimensional positioning result with three-dimensional information of any component in real time, thereby avoiding complex calculations, shortening the time consumption of indoor positioning, and improving the accuracy and reliability of the indoor positioning process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an indoor positioning method, device and electronic equipment. Background Art

[0002] Coal is the main energy source in my country and has played an important role in promoting the development of the national industry and the progress of the national economy. As a result, people have higher and higher requirements for the intelligent level of mining equipment and mining auxiliary equipment. In particular, the real-time monitoring, management and positioning of related equipment deployed indoors have become one of the most concerned topics.

[0003] However, there is no indoor positioning method that is time-saving, efficient and reliable in the related art. Therefore, how to accurately locate indoor related equipment has become a problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides an indoor positioning method, device and electronic device, which processes a first image frame with two-dimensional information based on a target indoor positioning model obtained by training with three-dimensional data, and can obtain a three-dimensional positioning result with three-dimensional information, thereby shortening the time consumption of indoor positioning and improving the accuracy and reliability of the indoor positioning process.

[0005] According to a first aspect of the present application, there is provided an indoor positioning method, comprising: acquiring a first image frame taken by an image acquisition device, wherein the image acquisition device is arranged on any component of any device in a target area; inputting the first image frame into a trained target indoor positioning model, and outputting a first three-dimensional positioning result of the any component corresponding to the first image frame.

[0006] In addition, an indoor positioning method according to the above embodiment of the present application may also have the following additional technical features:

[0007] According to one embodiment of the present application, inputting the first image frame into a trained target indoor positioning model and outputting a first three-dimensional positioning result of any component corresponding to the first image frame includes: inputting the first image frame into a feature extraction network in the target indoor positioning model, and having the feature extraction network perform feature extraction on the first image frame to obtain feature information of the first image frame; obtaining shooting parameters corresponding to when the image acquisition device shoots the first image frame based on the feature information; and obtaining the first three-dimensional positioning result of any component corresponding to the first image frame based on the shooting parameters.

[0008] According to one embodiment of the present application, after outputting the first three-dimensional positioning result of the device corresponding to the first image frame, it also includes: acquiring a second image frame captured by the image acquisition device, wherein the shooting time of the first image frame is inconsistent with the shooting time of the second image frame; inputting the second image frame into the target indoor positioning model, and outputting the second three-dimensional positioning result of any component of any device corresponding to the second image frame; and acquiring the movement trajectory of any component of any device corresponding to the image acquisition device based on the first three-dimensional positioning result and the second three-dimensional positioning result.

[0009] According to one embodiment of the present application, it also includes: acquiring third image frames captured by all the image acquisition devices set on any of the devices at the same time; inputting all the third image frames into the target indoor positioning model, and outputting the third three-dimensional positioning result of any of the devices.

[0010] According to one embodiment of the present application, the training process of the target indoor positioning model includes: obtaining three-dimensional environmental information of the target area, three-dimensional device information of at least one device in the target area, and sample shooting parameters of at least one image acquisition device set on any component of all the devices; obtaining the first sample image frames captured by all the image acquisition devices according to the corresponding sample shooting parameters; training the indoor positioning model to be trained according to the three-dimensional environmental information, the three-dimensional device information, the sample shooting parameters and the first sample image frames to obtain the target indoor positioning model.

[0011] According to one embodiment of the present application, the training of the indoor positioning model to be trained based on the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frame includes: inputting the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters into the indoor positioning model, and generating, by an image generation network in the indoor positioning model, image training values ​​captured by the image acquisition device according to the corresponding sample shooting parameters based on the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters; and performing feature extraction on the image training values ​​by a feature extraction network in the indoor positioning model to obtain feature information training values ​​of the image training values.

[0012] According to one embodiment of the present application, after the feature extraction network in the indoor positioning model performs feature extraction on the image training value to obtain the feature information training value of the image training value, it also includes: performing feature extraction on the first sample image frame to use the feature information of the first sample image frame as the true value of the feature information; obtaining the difference between the feature information training value and the true value of the feature information; adjusting the model parameters of the indoor positioning model according to the difference until the training end condition is met, and determining the indoor positioning model after the last adjustment of the model parameters as the target indoor positioning model.

[0013] According to a second aspect of the present application, an indoor positioning device is provided, comprising: an acquisition module, used to acquire a first image frame taken by an image acquisition device, wherein the image acquisition device is arranged on any component of any device in a target area; a positioning module, used to input the first image frame into a trained target indoor positioning model, and output a first three-dimensional positioning result of any component corresponding to the first image frame.

[0014] In addition, an indoor positioning device according to the above embodiment of the present application may also have the following additional technical features:

[0015] According to one embodiment of the present application, the positioning module is also used to: input the first image frame into a feature extraction network in the target indoor positioning model, and the feature extraction network performs feature extraction on the first image frame to obtain feature information of the first image frame; based on the feature information, obtain shooting parameters corresponding to when the image acquisition device shoots the first image frame; based on the shooting parameters, obtain the first three-dimensional positioning result of any component corresponding to the first image frame.

[0016] According to one embodiment of the present application, the positioning module is further used to: obtain a second image frame captured by the image acquisition device, wherein the shooting time of the first image frame is inconsistent with the shooting time of the second image frame; input the second image frame into the target indoor positioning model, and output a second three-dimensional positioning result of any component of any device corresponding to the second image frame; and obtain a movement trajectory of any component of any device corresponding to the image acquisition device based on the first three-dimensional positioning result and the second three-dimensional positioning result.

[0017] According to one embodiment of the present application, the acquisition module is also used to: acquire the third image frames taken by all the image acquisition devices set on any one of the devices at the same time; input all the third image frames into the target indoor positioning model, and output the third three-dimensional positioning result of any one of the devices.

[0018] According to one embodiment of the present application, it also includes: a training module, which is used to: obtain three-dimensional environmental information of the target area, three-dimensional device information of at least one device in the target area, and sample shooting parameters of at least one image acquisition device set on any component of all the devices; obtain the first sample image frames captured by all the image acquisition devices according to the corresponding sample shooting parameters; train the indoor positioning model to be trained according to the three-dimensional environmental information, the three-dimensional device information, the sample shooting parameters and the first sample image frames to obtain the target indoor positioning model.

[0019] According to one embodiment of the present application, the training module is also used to: input the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters into the indoor positioning model, and the image generation network in the indoor positioning model generates image training values ​​captured by the image acquisition device according to the corresponding sample shooting parameters based on the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters; and the feature extraction network in the indoor positioning model performs feature extraction on the image training values ​​to obtain feature information training values ​​of the image training values.

[0020] According to one embodiment of the present application, the training module is also used to: perform feature extraction on the first sample image frame to use the feature information of the first sample image frame as the true value of the feature information; obtain the difference between the feature information training value and the true value of the feature information; adjust the model parameters of the indoor positioning model according to the difference until the training end condition is met, and determine the indoor positioning model after the last adjustment of the model parameters as the target indoor positioning model.

[0021] In order to achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the aforementioned indoor positioning method is implemented.

[0022] In order to achieve the above-mentioned purpose, the fourth aspect embodiment of the present application proposes a non-temporary computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the aforementioned indoor positioning method is implemented.

[0023] In order to achieve the above-mentioned purpose, the fifth aspect of the present application proposes a computer program product, including a computer program, which, when executed by a processor, implements the indoor positioning method as described above.

[0024] The technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0025] The present application provides an indoor positioning method, which can obtain a first image frame taken by an image acquisition device, wherein the image acquisition device is set on any component of any equipment in a target area, and then input the first image frame into a trained target indoor positioning model, and output a first three-dimensional positioning result of any component corresponding to the first image frame, so that in the process of indoor positioning, only the first image frame with two-dimensional information can be relied upon, and the target indoor positioning model trained based on three-dimensional data such as indoor equipment appearance data and indoor environment data is processed. Optionally, a relative position relationship analysis is performed based on the first image frame to obtain a first three-dimensional positioning result with three-dimensional information of any component in real time, thereby avoiding complex calculations, shortening the time consumption of indoor positioning, and improving the accuracy and reliability of the indoor positioning process.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0028] Figure 1 A schematic diagram of a flow chart of an indoor positioning method provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of setting the position of an image acquisition device provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0031] Figure 4 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0032] Figure 5 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0033] Figure 6 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0034] Figure 7 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0035] Figure 8 A flowchart of another indoor positioning method provided in an embodiment of the present application;

[0036] Fig. 9 A schematic diagram of the structure of an indoor positioning device provided in an embodiment of the present application;

[0037] Fig.10 A schematic diagram of the structure of another indoor positioning device provided in an embodiment of the present application;

[0038] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0040] The indoor positioning method, device and electronic device of the present application are described in detail using embodiments below.

[0041] Figure 1 A flowchart of an indoor positioning method provided in an embodiment of the present application. It should be noted that the execution subject of the indoor positioning method of this embodiment is an indoor positioning device, which can be a hardware device or software in a hardware device. The hardware device is, for example, a terminal device, a server, etc.

[0042] like Figure 1 As shown, the indoor positioning method proposed in this embodiment includes the following steps:

[0043] S101. Acquire a first image frame captured by an image acquisition device, wherein the image acquisition device is disposed on any component of any device within a target area.

[0044] It should be noted that in actual applications, in order to conduct real-time monitoring, management and positioning of relevant equipment deployed indoors, image acquisition devices are usually arranged indoors to capture images of indoor equipment. However, after obtaining the captured images, it is necessary to rely on more complex methods to process the images before the devices displayed in the images can be located. Furthermore, unlike outdoor positioning, when positioning equipment in a relatively small indoor space, the accuracy requirements for positioning results are higher. As a result, the accuracy and reliability of indoor positioning results in related technologies often fail to meet expected requirements.

[0045] Therefore, the present application no longer relies solely on the images acquired by the image acquisition device, but proposes a time-saving, efficient and reliable indoor positioning method based on the relatively stable and easy-to-obtain characteristics of three-dimensional data such as indoor equipment appearance data and indoor environment data.

[0046] In the embodiment of the present application, an image acquisition device may be provided on any component of any device within the target area, and then a first image frame captured by the image acquisition device may be acquired.

[0047] Among them, the target area can optionally be any indoor area, for example, the target area can be workshop A; optionally, it can be any sub-area in any indoor area, for example, the target area can be sub-area A1 in workshop A.

[0048] Furthermore, the image acquisition device can be set at any position in the target area.

[0049] For example, for workshop A (target area), if Figure 2 As shown in (a), the image acquisition device can be arranged on the sliding component 2-1 of the device A; Figure 2 As shown in (b), the image acquisition device can be set on the robotic arm 2-2 of equipment A.

[0050] Among them, the image acquisition device can be any device for shooting, optionally, it can be a camera for acquiring images, such as a high-definition camera, etc.; optionally, it can be a camera for shooting videos, such as a panoramic camera, etc.

[0051] The first image frame may be any image frame captured by the image acquisition device. Optionally, a high-definition camera may be used to capture images one by one, and any image frame may be used as the first image frame; alternatively, a panoramic camera may be used to capture images, and the captured video may be framed to capture any image frame as the first image frame.

[0052] S102: Input the first image frame into a trained target indoor positioning model, and output a first three-dimensional positioning result of any component corresponding to the first image frame.

[0053] In an embodiment of the present application, after acquiring the first image frame, the first image frame may be input into a trained target indoor positioning model, and the target indoor positioning model may perform image processing on the first image frame to acquire a first three-dimensional positioning result.

[0054] The target indoor positioning model may be a converged model trained based on relatively stable and easily obtainable three-dimensional data such as indoor equipment appearance data and indoor environment data.

[0055] It should be noted that, since the image acquisition device is fixed on any component, relevant information of other devices, other components or other environments can be presented in the first image frame. In this case, the relevant information can be used as reference information for the positioning information of any device. In other words, by determining the relative positional relationship between other devices, other components or other environments and any component, the positioning of any component can be achieved to obtain the first three-dimensional positioning result.

[0056] The first three-dimensional positioning result may include at least one of the following sub-results: three-dimensional space coordinate information, angle information, etc. of any component.

[0057] The indoor positioning method provided by the present application can obtain a first image frame taken by an image acquisition device, wherein the image acquisition device is set on any component of any equipment in the target area, and then input the first image frame into a trained target indoor positioning model, and output a first three-dimensional positioning result of any component corresponding to the first image frame, so that in the process of indoor positioning, only the first image frame with two-dimensional information can be relied upon, and the target indoor positioning model trained based on three-dimensional data such as indoor equipment appearance data and indoor environment data is processed. Optionally, a relative position relationship analysis is performed based on the first image frame to obtain the first three-dimensional positioning result with three-dimensional information of any component in real time, thereby avoiding complex calculations, shortening the time consumption of indoor positioning, and improving the accuracy and reliability of the indoor positioning process.

[0058] It should be noted that in the present application, when attempting to input the first image frame into a trained target indoor positioning model and output the first three-dimensional positioning result of any component corresponding to the first image frame, the first image frame can be processed based on multiple networks in the target indoor positioning model.

[0059] As a possible implementation, Figure 3 As shown, based on the above embodiment, the specific process of inputting the first image frame into the trained target indoor positioning model in the above step S102 and outputting the first three-dimensional positioning result of any component corresponding to the first image frame includes the following steps:

[0060] S201. Input a first image frame into a feature extraction network in a target indoor positioning model, and use the feature extraction network to extract features from the first image frame to obtain feature information of the first image frame.

[0061] It should be noted that feature extraction is performed on the first image frame, that is, image feature extraction is performed on the first image frame; the feature information of the first image frame may include image feature information such as brightness, edge, texture, color, and histogram.

[0062] It should be noted that the present application does not limit the specific method of extracting features from the first image frame by the feature extraction network, and it can be set according to actual conditions.

[0063] As a possible implementation manner, color features, texture features, algebraic features, and transformation features of the first image frame may be extracted to obtain feature information of the first image frame.

[0064] Optionally, feature extraction may be performed on the first image frame to obtain feature information based on the number of single colors, the center position of a single color, edge detection, finding the diameter and center position of a circle using Hough circle transform, finding the diameter and center position of a circle using least squares method, etc.

[0065] S202: Acquire shooting parameters corresponding to when the image acquisition device shoots the first image frame according to the feature information.

[0066] It should be noted that the target indoor positioning model can simulate the relevant parameters when the image acquisition device acquires the first image frame. Optionally, the relevant parameters may include shooting parameters of the image acquisition device, environmental information carried in the image captured by the image acquisition device according to the shooting parameters, equipment information and other parameters. In other words, the target indoor positioning model can simulate the acquisition process of the image acquisition device acquiring the first image frame.

[0067] The shooting parameters include at least the exposure settings, mounting positions, angles and other parameters of the image acquisition parameters.

[0068] S203: Obtain a first three-dimensional positioning result of any component corresponding to the first image frame according to the shooting parameters.

[0069] In the embodiment of the present application, after obtaining the shooting parameters, the three-dimensional positioning information of any component can be inferred based on the shooting parameters.

[0070] For example, for the feature information of the first image frame obtained, the corresponding shooting parameter x={a, b, c, ...} when the image acquisition device takes the first image frame can be simulated based on the feature information. Further, based on the shooting parameter x, the first three-dimensional positioning result of any component corresponding to the first image frame taken by the image acquisition device can be inferred.

[0071] The indoor positioning method provided in the present application can input the first image frame into a feature extraction network in the target indoor positioning model, and the feature extraction network can extract features from the first image frame to obtain feature information of the first image frame, and then obtain shooting parameters corresponding to the image acquisition device when shooting the first image frame based on the feature information, and then obtain a first three-dimensional positioning result of any component corresponding to the first image frame based on the shooting parameters, so that the first three-dimensional positioning result of any component corresponding to the first image frame can be reversely inferred based on the shooting parameters, environmental information, equipment information and other parameters carried in the first image frame, thereby further improving the accuracy and reliability of the indoor positioning process.

[0072] It should be noted that, in the present application, after obtaining the first three-dimensional positioning result, any other frame of image acquired by the image acquisition device can be acquired, and the movement trajectory of any component can be acquired based on the two frames of images acquired at different times.

[0073] As a possible implementation, Figure 4 As shown, based on the above embodiment, the specific process after outputting the first three-dimensional positioning result of the device corresponding to the first image frame in the above step includes the following steps:

[0074] S301. Acquire a second image frame captured by an image acquisition device, wherein the capturing time of the first image frame is inconsistent with the capturing time of the second image frame.

[0075] It should be noted that when any component moves, the image acquisition device also moves, and in this case, the shooting parameters of the image acquisition device also change. Therefore, by acquiring the second image frame captured by the image acquisition device, the movement trajectory of any component can be obtained.

[0076] S302: Input the second image frame into the target indoor positioning model, and output a second three-dimensional positioning result of any component of any device corresponding to the second image frame.

[0077] In an embodiment of the present application, after acquiring the second image frame, the second image frame can be input into a feature extraction network in the target indoor positioning model, and the feature extraction network performs feature extraction on the second image frame to obtain feature information of the second image frame, and then based on the feature information, the corresponding shooting parameters when the image acquisition device takes the second image frame are obtained, and then based on the shooting parameters, the second three-dimensional positioning result of any component corresponding to the second image frame is obtained.

[0078] S303: Acquire a moving trajectory of any component of any device corresponding to the image acquisition device according to the first three-dimensional positioning result and the second three-dimensional positioning result.

[0079] The moving trajectory at least includes information such as the moving distance, moving direction, and rotation angle of any component.

[0080] Furthermore, after obtaining the movement trajectory of any component within the time interval between shooting the first image frame and the second image frame, the movement trajectories of all other components of any device corresponding to any component within the same time interval can be obtained, and then the device can be positioned based on the movement trajectories of all components, and the movement trajectory of the device can be determined.

[0081] As a possible implementation, Figure 5 As shown, based on the above embodiment, the following steps are specifically included:

[0082] S401: Acquire a third image frame captured by all image acquisition devices provided on any device at the same time.

[0083] It should be noted that in the present application, at least one image acquisition device is provided on any component. In this way, if an attempt is made to locate n components of any device, the third image frame taken by the n image acquisition devices provided on any device at the same time can be obtained.

[0084] S402: Input all third image frames into the target indoor positioning model, and output the third three-dimensional positioning result of any device.

[0085] In an embodiment of the present application, after acquiring the third image frame, the third image frame can be input into a feature extraction network in the target indoor positioning model, and the feature extraction network performs feature extraction on the third image frame to obtain feature information of the third image frame, and then based on the feature information, the corresponding shooting parameters when the image acquisition device takes the third image frame are obtained, and then based on the shooting parameters, the third three-dimensional positioning result of any component corresponding to the third image frame is obtained.

[0086] It should be noted that the present application does not limit the specific method of inputting the third image frame into the target indoor positioning model, and it can be set according to actual conditions.

[0087] As a possible implementation manner, optionally, any third image frame may be input in sequence; optionally, all third image frames may be input into the target indoor positioning model at the same time.

[0088] The following uses an embodiment to explain the training process of the target indoor positioning model of the present application.

[0089] As a possible implementation, Figure 6 As shown, based on the above embodiment, the following steps are specifically included:

[0090] S501, acquiring three-dimensional environment information of a target area, three-dimensional device information of at least one device in the target area, and sample shooting parameters of at least one image acquisition device set on any component of any device.

[0091] It should be noted that, in the present application, the training process for the target indoor positioning model is equivalent to simulating the shooting environment in a real scene.

[0092] Therefore, in order to improve training accuracy, the three-dimensional environmental information of all sub-areas of the target area, the three-dimensional device information of all devices in the target area, and the sample shooting parameters of all image acquisition devices set on all components of each device can be obtained.

[0093] Furthermore, multiple image acquisition devices may be arranged on the same component.

[0094] S502: Acquire the first sample image frame captured by all image acquisition devices according to corresponding sample capturing parameters.

[0095] Step S502 is consistent with step S101 and will not be described again here.

[0096] S503: Train the indoor positioning model to be trained according to the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frame to obtain a target indoor positioning model.

[0097] As a possible implementation, Figure 7 As shown, based on the above embodiment, the specific process of training the indoor positioning model to be trained according to the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frame in the above step S503 includes the following steps:

[0098] S601. Input three-dimensional environment information, three-dimensional device information and sample shooting parameters into the indoor positioning model, and the image generation network in the indoor positioning model generates image training values ​​captured by the image acquisition device according to the corresponding sample shooting parameters based on the three-dimensional environment information, three-dimensional device information and sample shooting parameters.

[0099] S602: A feature extraction network in the indoor positioning model performs feature extraction on the image training value to obtain a feature information training value of the image training value.

[0100] It should be noted that in actual applications, the computers used to train the model may have different performance, such as different computing power. In this way, for computers with higher computing power, after obtaining the first sample image, a preset time step can be pre-set, and then the model training can be performed according to the preset time step.

[0101] Optionally, in the process of acquiring the characteristic information training value, the movement direction and distance of the virtual image acquisition device after the next preset time step is completed may be predicted, that is, the possible position of any component may be predicted.

[0102] Optionally, image frames obtained after different rotation directions, different rotation angles, different translation directions, and different translation distances of any component can be predicted to obtain at least one prediction result. Furthermore, all the prediction results can be further processed, and one of the prediction results can be selected from all the prediction results as the feature information training value.

[0103] Furthermore, after acquiring the feature information training value of the image training value, the feature information true value may be acquired, and the feature information true value may be compared with the feature information training value to adjust the model parameters of the indoor positioning model.

[0104] As a possible implementation, Figure 8 As shown, based on the above embodiment, the following steps are specifically included:

[0105] S701: Extract features from a first sample image frame to use feature information of the first sample image frame as a true value of the feature information.

[0106] In the embodiment of the present application, feature extraction is performed on the first sample image frame, that is, feature extraction is performed on the first sample image frame captured by a real image acquisition device to obtain a unique true value of the feature information.

[0107] S702: Obtain the difference between the feature information training value and the feature information true value.

[0108] In the embodiment of the present application, after obtaining the true value of the feature information, the feature information training value may be compared with the true value of the feature information to obtain the difference between the feature information training value and the true value of the feature information.

[0109] S703: Adjust the model parameters of the indoor positioning model according to the difference until the training end condition is met, and determine the indoor positioning model after the model parameters are adjusted for the last time as the target indoor positioning model.

[0110] In an embodiment of the present application, when the difference between the feature information training value and the true value of the feature information is obtained, the model parameters can be adjusted according to the difference so that the feature information training value is close to the expected output of the model. Optionally, the input layer data can be nonlinearly transformed through the Sigmoid activation function, and the weight Wi can be adjusted in the hidden layer so that the feature information training value is close to the expected output of the model.

[0111] Further, after adjusting the model parameters of the indoor positioning model to be trained according to the difference, when the training end condition is met, the indoor positioning model to be trained after the model parameters are adjusted for the last time can be determined as the target indoor positioning model.

[0112] The training stop condition can be set according to the actual situation. For example, the training stop condition can be set to the difference between the feature information training value and the feature information true value is less than the preset training difference value threshold; for another example, the training stop condition can be set to the number of times the model parameter is adjusted reaches the preset number threshold.

[0113] The indoor positioning method provided in the present application is based on a model training method with high reliability, which ensures the training effect of the target indoor positioning model, improves the accuracy, reliability and effectiveness of the output three-dimensional positioning results, and lays the foundation for accurately obtaining three-dimensional positioning results.

[0114] To sum up, the indoor positioning method proposed in the present application can calculate the position of the virtual positioned object and process the video data of the virtual image acquisition device based on the target indoor positioning model, and obtain the possible values ​​(prediction results) of the feature quantities in the virtual video corresponding to the different rotation directions, translation directions, rotation angles, and translation distances of the relevant components in the next time step.

[0115] At the same time, by comparing the video data processed by the actual image acquisition device to obtain its characteristic values, the accurate rotation direction, translation direction, rotation angle, and translation distance of the actual positioned object in the next time step are obtained, thereby providing a method for accurately determining the actual position and spatial state of equipment and components in target areas such as workshops without using sensors.

[0116] Furthermore, since the computational complexity of the data feature extraction process of the virtual image acquisition device is much lower than that of the data feature extraction process of the physical image acquisition device, the reason is that the virtual image acquisition device can obtain pure video data without any external interference. Therefore, the processing speed of the virtual image acquisition device is much faster than that of the actual image acquisition device.

[0117] Furthermore, by comparing the video feature values ​​of the virtual located object with the video feature values ​​of the real located object, the image acquisition device or the bracket status of the image acquisition device is identified, which is equivalent to allowing the computer to recognize the three-dimensional information of the object to be identified in advance. Compared with the video recognition of the position information of the real located object, the amount of calculation is greatly reduced and the calculation speed is greatly improved.

[0118] In order to implement the above embodiment, this embodiment provides an indoor positioning device. Fig. 9 A schematic diagram of the structure of an indoor positioning device provided in an embodiment of the present application.

[0119] like Fig. 9 As shown, the indoor positioning device 1000 includes: an acquisition module 110 and a positioning module 120. Among them,

[0120] An acquisition module 110 is used to acquire a first image frame captured by an image acquisition device, wherein the image acquisition device is disposed on any component of any device in the target area;

[0121] The positioning module 120 is used to input the first image frame into a trained target indoor positioning model, and output a first three-dimensional positioning result of any component corresponding to the first image frame.

[0122] According to an embodiment of the present application, the positioning module 120 is further used to: input the first image frame into a feature extraction network in the target indoor positioning model, and have the feature extraction network perform feature extraction on the first image frame to obtain feature information of the first image frame; based on the feature information, obtain shooting parameters corresponding to when the image acquisition device shoots the first image frame; based on the shooting parameters, obtain the first three-dimensional positioning result of any component corresponding to the first image frame.

[0123] According to an embodiment of the present application, the positioning module 120 is further used to: obtain a second image frame captured by the image acquisition device, wherein the shooting time of the first image frame is inconsistent with the shooting time of the second image frame; input the second image frame into the target indoor positioning model, and output a second three-dimensional positioning result of any component of any device corresponding to the second image frame; and obtain a moving trajectory of any component of any device corresponding to the image acquisition device based on the first three-dimensional positioning result and the second three-dimensional positioning result.

[0124] According to an embodiment of the present application, the acquisition module 110 is also used to: acquire the third image frames taken by all the image acquisition devices set on any of the devices at the same time; input all the third image frames into the target indoor positioning model, and output the third three-dimensional positioning result of any of the devices.

[0125] According to the embodiment of the present application, Fig.10As shown, the indoor positioning device 1000 further includes: a training module 130. The training module 130 is used to: obtain the three-dimensional environment information of the target area, the three-dimensional device information of at least one device in the target area, and the sample shooting parameters of at least one image acquisition device set on any component of all the devices; obtain the first sample image frames captured by all the image acquisition devices according to the corresponding sample shooting parameters; train the indoor positioning model to be trained according to the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frames to obtain the target indoor positioning model.

[0126] According to the embodiment of the present application, the training module 130 is further used to: input the 3D environment information, the 3D device information and the sample shooting parameters into the indoor positioning model, and the image generation network in the indoor positioning model generates the image training value captured by the image acquisition device according to the corresponding sample shooting parameters according to the 3D environment information, the 3D device information and the sample shooting parameters; and the feature extraction network in the indoor positioning model extracts features from the image training value to obtain the feature information training value of the image training value. .

[0127] According to an embodiment of the present application, the training module 130 is further used to: perform feature extraction on the first sample image frame to use the feature information of the first sample image frame as the true value of the feature information; obtain the difference between the feature information training value and the true value of the feature information; adjust the model parameters of the indoor positioning model according to the difference until the training end condition is met, and determine the indoor positioning model after the last adjustment of the model parameters as the target indoor positioning model.

[0128] According to the indoor positioning device provided by the present application, a first image frame captured by an image acquisition device can be acquired, wherein the image acquisition device is arranged on any component of any equipment in the target area, and then the first image frame is input into a trained target indoor positioning model, and a first three-dimensional positioning result of any component corresponding to the first image frame is output, so that in the process of indoor positioning, only the first image frame with two-dimensional information can be relied upon, and the target indoor positioning model trained based on three-dimensional data such as indoor equipment appearance data and indoor environment data is used for processing. Optionally, a relative position relationship analysis is performed based on the first image frame to obtain the first three-dimensional positioning result with three-dimensional information of any component in real time, thereby avoiding complex calculations, shortening the time consumption of indoor positioning, and improving the accuracy and reliability of the indoor positioning process.

[0129] In order to implement the above embodiment, the present application also proposes an electronic device 3000, such as Fig.11As shown, it includes a memory 310, a processor 320, and a computer program stored in the memory 310 and executable on the processor 320. When the processor executes the program, the aforementioned indoor positioning method is implemented.

[0130] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned indoor positioning method is implemented.

[0131] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, and when the computer program is executed by a processor, the indoor positioning method as described above is implemented.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0133] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An indoor positioning method, include: Acquire a first image frame captured by an image acquisition device, wherein the image acquisition device is disposed on any component of any device in the target area; Inputting the first image frame into a trained target indoor positioning model, and outputting a first three-dimensional positioning result of any component corresponding to the first image frame; The step of inputting the first image frame into a trained target indoor positioning model and outputting a first three-dimensional positioning result of any component corresponding to the first image frame includes: Inputting the first image frame into a feature extraction network in the target indoor positioning model, and having the feature extraction network perform feature extraction on the first image frame to obtain feature information of the first image frame; According to the characteristic information, acquiring shooting parameters corresponding to when the image acquisition device shoots the first image frame; Acquire the first three-dimensional positioning result of any one of the components corresponding to the first image frame according to the shooting parameters; Wherein, after outputting the first three-dimensional positioning result of the device corresponding to the first image frame, the method further includes: Acquire a second image frame captured by the image acquisition device, wherein the capturing time of the first image frame is inconsistent with the capturing time of the second image frame; Inputting the second image frame into the target indoor positioning model, and outputting a second three-dimensional positioning result of any component of any device corresponding to the second image frame; Acquire a moving track of any component of any device corresponding to the image acquisition device according to the first three-dimensional positioning result and the second three-dimensional positioning result; The training process of the target indoor positioning model includes: Acquire three-dimensional environmental information of a target area, three-dimensional device information of at least one device in the target area, and sample shooting parameters of at least one image acquisition device set on any component of all the devices; Acquire the first sample image frames captured by all the image acquisition devices according to the corresponding sample shooting parameters; Training the indoor positioning model to be trained according to the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frame to obtain the target indoor positioning model; The step of training the indoor positioning model to be trained according to the three-dimensional environment information, the three-dimensional device information, the sample shooting parameters and the first sample image frame includes: The three-dimensional environment information, the three-dimensional device information and the sample shooting parameters are input into the indoor positioning model, and the image generation network in the indoor positioning model generates the image training value captured by the image acquisition device according to the corresponding sample shooting parameters based on the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters; The feature extraction network in the indoor positioning model performs feature extraction on the image training value to obtain a feature information training value of the image training value.

2. The positioning method according to claim 1, in, Also includes: Acquire a third image frame captured at the same time by all the image acquisition devices set on any one of the devices; All of the third image frames are input into the target indoor positioning model, and a third three-dimensional positioning result of any of the devices is output.

3. The positioning method according to claim 1, in, After the feature extraction network in the indoor positioning model extracts features from the image training value to obtain the feature information training value of the image training value, the method further includes: Performing feature extraction on the first sample image frame to use feature information of the first sample image frame as a true value of feature information; Obtaining a difference between the feature information training value and the feature information true value; The model parameters of the indoor positioning model are adjusted according to the difference until a training end condition is met, and the indoor positioning model after the model parameters are adjusted for the last time is determined as the target indoor positioning model.

4. An indoor positioning device, include: An acquisition module, used for acquiring a first image frame captured by an image acquisition device, wherein the image acquisition device is disposed on any component of any device in the target area; A positioning module, used for inputting the first image frame into a trained target indoor positioning model, and outputting a first three-dimensional positioning result of any component corresponding to the first image frame; The positioning module is further used to: input the first image frame into a feature extraction network in the target indoor positioning model, and the feature extraction network performs feature extraction on the first image frame to obtain feature information of the first image frame; obtain shooting parameters corresponding to when the image acquisition device shoots the first image frame according to the feature information; and obtain the first three-dimensional positioning result of any component corresponding to the first image frame according to the shooting parameters; The positioning module is further used to: obtain a second image frame captured by the image acquisition device, wherein the shooting time of the first image frame is inconsistent with the shooting time of the second image frame; input the second image frame into the target indoor positioning model, and output a second three-dimensional positioning result of the any component of the any device corresponding to the second image frame; and obtain a moving trajectory of the any component of the any device corresponding to the image acquisition device according to the first three-dimensional positioning result and the second three-dimensional positioning result; A training module is used to: obtain three-dimensional environmental information of a target area, three-dimensional device information of at least one device in the target area, and sample shooting parameters of at least one image acquisition device set on any component of all the devices; obtain first sample image frames captured by all the image acquisition devices according to the corresponding sample shooting parameters; train an indoor positioning model to be trained according to the three-dimensional environmental information, the three-dimensional device information, the sample shooting parameters and the first sample image frames to obtain the target indoor positioning model; The training module is also used to: input the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters into the indoor positioning model, and the image generation network in the indoor positioning model generates image training values ​​captured by the image acquisition device according to the corresponding sample shooting parameters based on the three-dimensional environment information, the three-dimensional device information and the sample shooting parameters; and the feature extraction network in the indoor positioning model performs feature extraction on the image training values ​​to obtain feature information training values ​​of the image training values.

5. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the indoor positioning method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the indoor positioning method as described in any one of claims 1 to 3 is implemented.

Citation Information

Patent Citations

  • Positioning method and device and mobile terminal

    CN108009588A

  • Intelligent indoor positioning system based on icon recognition and positioning method thereof

    CN109341691A

  • Trajectory prediction method and device, readable storage medium and electronic equipment

    CN110751683A

  • Method and device for identifying space target and storage medium

    CN111598951A

  • Method for realizing movement track estimation by using picture and related device

    CN112118537A