Spatial positioning method, device, electronic device and storage medium

Abstract

The present disclosure relates to a spatial positioning method, device, electronic device and storage medium, and relates to the field of spatial positioning. The method includes: obtaining at least one image of a target object in a space station; obtaining first positioning information of the target object in a body area network; determining target positioning information of at least one node of the target object based on at least one image, the first positioning information and a preset first positioning model and a second positioning model; and positioning the target object in the space station based on the target positioning information. The application of the present disclosure can fuse the image of the target object in the space station with the body area network positioning information, thereby improving the accuracy of positioning the object in the space station.

Description

Technical Field

[0001] The present application relates to the field of space positioning technology, and in particular to a space positioning method, device, electronic device and storage medium. Background Art

[0002] On Earth, satellite positioning systems and ultra-wideband (UWB) positioning technology have solved the problem of indoor and outdoor positioning for both military and civilian use with absolute positioning accuracy. However, for positioning inside and outside the space station cabin, due to signal shielding and interference from the complex electromagnetic environment in space, conventional positioning technology is obviously not applicable. Due to its special working environment, it is urgent to establish an effective indoor and outdoor personnel positioning system for the space station cabin, which is of great significance for astronaut identification, space station equipment safety management and space station safety avoidance. Summary of the invention

[0003] Embodiments of the present disclosure provide a spatial positioning method, device, electronic device, and storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a spatial positioning method, including: acquiring at least one image of a target object in a space station; acquiring first positioning information of the target object in a body area network; determining target positioning information of at least one node of the target object based on at least one image, the first positioning information, and a preset first positioning model and a second positioning model; and positioning the target object in the space station based on the target positioning information.

[0005] In a second aspect, an embodiment of the present disclosure provides a spatial positioning device, comprising: a first acquisition unit, configured to acquire at least one image of a target object in a space station; a second acquisition unit, configured to acquire first positioning information of the target object in a body area network; a position determination unit, configured to determine target positioning information of at least one node of the target object based on at least one image, the first positioning information, and a preset first positioning model and a second positioning model; and the spatial positioning unit, configured to position the target object in the space station based on the target positioning information.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein the processor implements the spatial positioning method described in the first aspect when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spatial positioning method as described in the first aspect.

[0008] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0010] Figure 1 is an exemplary system architecture diagram to which an embodiment of the spatial positioning method disclosed herein may be applied;

[0011] Figure 2 A schematic diagram of a flow chart of an embodiment of the spatial positioning method disclosed in the present invention;

[0012] Figure 3 A schematic diagram of an application scenario of the spatial positioning method disclosed in the present invention;

[0013] Figure 4 A schematic diagram of a flow chart of another embodiment of the spatial positioning method disclosed in the present invention;

[0014] Figure 5 It is a structural schematic diagram of an embodiment of the space positioning device disclosed in the present invention;

[0015] Figure 6 A schematic diagram of the structure of an embodiment of an electronic device disclosed herein. DETAILED DESCRIPTION

[0016] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0017] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0018] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0019] In order to make the technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0020] Figure 1An exemplary system architecture 100 is shown to which embodiments of the spatial positioning method or spatial positioning device of the present disclosure can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include a body area network positioning base station 101, image acquisition devices 102, 103 and a terminal device 104. The body area network positioning base station 101, the image acquisition devices 102, 103 and the terminal 104 may communicate through various space communication technologies.

[0022] The body area network positioning base station 101 can be set in the space station, which can send signals in the human body communication (HBC) frequency band to the outside through a prefabricated chip to form a human body sensor network (also known as a body area network). The human body sensor network is a human body capacitive coupling communication technology first proposed in the 1990s. Its essence is based on the characteristics of the human body receiving and transmitting electromagnetic waves, that is, using the electromagnetic effect of the human body to use the "human body" as a medium for transmitting information, thereby transmitting information. It has the advantages of high security and low power consumption, and can meet the transmission of various information. The human body sensor network uses the human body as a transmission medium for wireless communication to achieve the purpose of communication. Its physical properties determine that this technology can achieve real-time, high-precision, and high-security positioning and communication. It can measure the flight time of radio signals very accurately, and can achieve centimeter-level precise distance or position measurement at most. Due to the characteristic of human body coupling electromagnetic waves, the energy range is centered on the human body, and its accuracy can reach 10cm. At the same time, due to its lower frequency and better penetration, the ranging range is greater than that of UWB short pulses, and the data transmission rate at the physical layer (PHY) can reach 164kbit / s-1.3Mbit / s.

[0023] The image acquisition devices 102 and 103 can be arranged at different positions in the space station to acquire real-time images of astronauts in the space station. Here, the image acquisition devices 102 and 103 can be devices with processing chips, which can perform various processing on the images immediately after acquiring the real-time images, and can also immediately send the acquired images to the terminal device 104 on the ground through space communication technology.

[0024] The terminal device 104 can be respectively set up in the space station and the ground command center to process the astronaut's body area network signals and images to locate the astronaut, and can also send information or instructions based on the astronaut's position, etc.

[0025] The terminal device 104 can be hardware or software. When the terminal device 104 is hardware, it can be various electronic devices, including but not limited to tablet computers, laptop computers, and desktop computers. When the terminal device 104 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be noted that the space positioning method provided in the embodiment of the present disclosure is generally executed by the terminal device 104. Accordingly, the space positioning device is generally disposed in the terminal device 104.

[0027] It should be understood that Figure 1 The number of body area network positioning base stations, image acquisition devices and terminal devices in the figure is only illustrative. According to the implementation requirements, there can be any number of body area network positioning base stations, image acquisition devices and terminal devices.

[0028] Figure 2 FIG. 2 shows a process 200 of an embodiment of the spatial positioning method of the present disclosure. Figure 2 As shown, the spatial positioning method of this embodiment may include the following steps:

[0029] Step 201, obtaining at least one image of a target object in a space station.

[0030] In this embodiment, the execution subject of the spatial positioning method (for example Figure 1 The terminal device 104 shown in the figure can obtain at least one image of the target object in the space station. Specifically, the execution subject can communicate with at least one image acquisition device (e.g. Figure 1 The at least one image acquisition device 102, 103 shown in the figure can be arranged at different positions of the space station to acquire images of the target object from different angles. The target object can be various objects in the space station, for example, an astronaut.

[0031] In some optional implementations of this embodiment, the at least one image may be an image captured by different image acquisition devices at the same time. Specifically, each image acquisition device may capture an image sequence of the target object at the same frequency and in real time. The images captured at the same time are further processed as target images. In some specific practices, the image acquisition device or the execution subject may pre-process the target image, for example, the target image may be compressed, cropped, etc.

[0032] Step 202: Acquire first positioning information of the target object in the body area network.

[0033] The execution subject can also obtain the first positioning information of the target object in the body area network. Here, the body area network is a network attached to the human body, which consists of a set of small, movable sensors with communication functions and a body area positioning base station (or BAN coordinator). Each sensor can be worn on the body or implanted in the body. The body area positioning base station can send signals of a preset frequency band to the outside, and the sensors installed on the human body can receive the above signals and send signals to the outside. The human body can be positioned by the signals radiated outward by each sensor to obtain the first positioning information. Here, the first positioning information can represent the position of different nodes of the target object in the space station, which can be in the form of a string or a coordinate.

[0034] Step 203: Determine target positioning information of at least one node of the target object according to at least one image, the first positioning information, and the preset first positioning model and second positioning model.

[0035] In this embodiment, after acquiring the above-mentioned at least one image and the first positioning information, the execution subject can combine the preset first positioning model and the second positioning model to determine the target positioning information of at least one node of the target object. Here, the first positioning model and the second positioning model can be various deep learning models. Specifically, the first positioning model can be used to process the image and output a positioning information. The second positioning model can be used to process the first positioning information and also output a positioning information. The execution subject can combine the two positioning information to obtain the target positioning information. Alternatively, the first positioning model can process the image and the positioning information at the same time and output a positioning information. The second positioning model can further process the positioning model output by the first positioning model to obtain the target positioning information. In some specific practices, the first positioning model can be a recurrent neural network (RNN), and the second positioning model can be a stacked hourglass network (Stacked Hourglass Network).

[0036] In this embodiment, the target object may include multiple nodes. Here, the node may be a joint point of the target object. The execution subject may first perform joint point recognition on the image to determine the position of the joint point. Then, the position of each joint point may be further input into the first positioning model and / or the second positioning model to determine the target positioning information of each node. Here, the target positioning information may be the coordinate information of each node.

[0037] Step 204: Position the target object in the space station according to the target positioning information.

[0038] After obtaining the target positioning information, the execution subject can analyze it to determine the position and / or posture of an object in the space station.

[0039] Continue to see Figure 3 , which shows a schematic diagram of an application scenario of the spatial positioning method according to the present disclosure. Figure 3 In the application scenario, the astronauts are active in the space station, wearing sensor devices. The auxiliary cameras 1 and 2 in the space station collect images of the astronauts in real time. At the same time, the body area network positioning base station sends signals to the outside, and the sensor devices on the astronauts receive the above signals and send signals conducted by the human body to the outside. The terminal device in the space station receives the images collected in real time by the auxiliary cameras 1 and 2 on the one hand, and receives the signals sent by the sensor devices on the astronauts on the other hand. The terminal device parses the signals sent by the sensor devices to obtain the first positioning information. At the same time, the images collected in real time by the auxiliary cameras 1 and 2 are input into the pre-trained recurrent neural network to obtain the second positioning information of the astronauts. Then the first positioning information and the second positioning information are input into the pre-trained stacked hourglass network to obtain the target positioning information of the astronauts, thereby realizing the positioning of the astronauts in the space station.

[0040] The spatial positioning method provided by the above-mentioned embodiments of the present disclosure can fuse the image of the target object in the space station with the body area network positioning information, thereby improving the accuracy of positioning the object in the space station.

[0041] Continue to see Figure 4 , which shows a process 400 of another embodiment of the spatial positioning method according to the present disclosure. Figure 4 As shown, the method in this embodiment may include the following steps:

[0042] Step 401, obtaining an image set collected by at least one image collection device installed at a preset position in the space station; and selecting images collected by different image collection devices at the same time from the image set.

[0043] In this embodiment, at least one image acquisition device may be installed at a preset position in the space station. Each of the above image acquisition devices is used to acquire images in the space station in real time to obtain an image set. Then, the execution subject may select images acquired by different image acquisition devices at the same time from the above image set. In this way, the selected images may include information of different angles of the target object.

[0044] Step 402, obtaining a body area network signal of the target object, wherein the body area network signal is sent outward after the target object receives a baseband signal sent by a body area network positioning base station set in the space station; parsing the body area network signal to determine the first positioning information.

[0045] In this embodiment, a body area network positioning base station is also provided in the space station. The body area network positioning base station can send baseband signals to the outside. The target object can wear multiple sensors, and specifically, the above sensors can be set at each node of the target object. The execution subject can receive the body area network signal sent by the above sensors. And the above body area network signal is analyzed to determine the first positioning information. Here, the analysis of the body area network signal is a prior art and will not be repeated here.

[0046] Step 403: fuse the at least one image to obtain a fused image.

[0047] The execution subject may fuse at least one of the above images to obtain a fused image. Specifically, the execution subject may use an existing registration algorithm to register the above images, and then splice them after registration. The obtained image is called a fused image. Alternatively, the execution subject may first extract the features of each image and calculate the similarity between the features of each image. When the similarity is greater than a preset threshold, the features are fused and calculated to obtain a fused feature vector. The fused feature vector is further input into a pre-trained deep learning model to obtain a fused image.

[0048] In some optional implementations of this embodiment, the execution entity can achieve image fusion through the following steps: extracting feature points of the target object in at least one image; registering at least one image based on the feature points; and fusing at least one registered image to obtain a fused image.

[0049] In this implementation, the execution subject may first extract the feature points of the target object in at least one image. Here, the existing feature point extraction algorithm may be used to realize the extraction of feature points. Then, the execution subject may use algorithms such as nearest neighbor and next nearest neighbor to match the feature points, select images with higher quality (that can extract clear feature points) for splicing, and discard images with lower quality. The images are registered according to the extracted feature points. Specifically, the execution subject may overlap the feature points representing the same position in each image to realize the image registration. Then, the execution subject may fuse the registered images. Specifically, the OpenCV transformer may be used to deform the image, and then the transformed image may be compensated to generate a fused image.

[0050] Step 404: Determine second positioning information of at least one node of the target object according to the fused image and the first positioning model.

[0051] After obtaining the fused image, the execution subject may input the fused image into the first positioning model. Here, the first positioning model is used to characterize the correspondence between the fused image and the positioning information. Specifically, the first positioning model may be a recurrent neural network. The second positioning information may be a set of strings. Each string may represent the location information of each node of the target object.

[0052] Step 405: Determine target positioning information of at least one node according to the first positioning information, the second positioning information and the second positioning model.

[0053] After obtaining the first positioning information and the second positioning information, the execution subject may combine the two. Here, the second positioning information may be spliced ​​after the first positioning information to obtain the third positioning information. Alternatively, the execution subject may take the intersection of the areas summarized by the first positioning information and the second positioning information as the third positioning information.

[0054] Then, the third positioning information can be input into the second positioning model. Here, the second positioning model is used to characterize the relationship between the positioning information and the node positioning information. The output of the second positioning model is the target positioning information of each node of the target object. Here, the target positioning information can be a coordinate or a connection diagram between nodes.

[0055] In some optional implementations of this embodiment, the execution entity may determine the target positioning information through the following steps: combining the first positioning information and the second positioning information to obtain third positioning information; determining a connection diagram of at least one node based on the third positioning information and the second positioning model; and determining the target positioning information based on the connection diagram.

[0056] In this implementation, the execution subject may combine the first positioning information and the second positioning information to obtain the third positioning information. The combination here may be splicing. The execution subject may directly input the third positioning information into the second positioning model to determine the connection diagram of at least one node. Here, the second positioning model may be a modified stacked hourglass network. Finally, the execution subject may determine the target positioning information based on the connection diagram. Specifically, the execution subject may delineate the outline of the connection diagram, and the area delineated by the above outline is the location of the target object.

[0057] In some optional implementations of this embodiment, the execution subject can determine the head area of ​​the target object based on the connection diagram. Then, the posture of the target object is determined based on the head area. Specifically, each node of the target object may include the key points of the neck and the key points of the two ears of the target object. The line between the key points of the neck and the two ears of the face can be used as the head area. After determining the head area, the direction of the target object can be determined. Then, the posture of the target object can be determined in combination with other areas in the connection diagram.

[0058] Step 406: Position the target object in the space station according to the target positioning information.

[0059] The spatial positioning method provided by the above-mentioned embodiments of the present disclosure can utilize multi-point computer-assisted image acquisition, combined with a deep reinforcement learning recurrent neural network model, to perform secondary positioning of astronauts in the space station, and integrate body area network positioning information to further make the positioning more accurate.

[0060] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a spatial positioning device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0061] like Figure 5 As shown, the space positioning device 500 of this embodiment includes: a first acquisition unit 501 , a second acquisition unit 502 , a position determination unit 503 and a space positioning unit 504 .

[0062] The first acquisition unit 501 is configured to acquire at least one image of a target object in the space station.

[0063] The second acquiring unit 502 is configured to acquire first positioning information of the target object in the body area network.

[0064] The position determination unit 503 is configured to determine target positioning information of at least one node of the target object according to at least one image, the first positioning information, and a preset first positioning model and a second positioning model.

[0065] The space positioning unit 504 is configured to position the target object in the space station according to the target positioning information.

[0066] In addition, in the technical solution of the present application, an electronic device is also proposed.

[0067] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0068] like Figure 6As shown, the electronic device may include a processor 601, a memory 602, a bus 603, and a computer program stored in the memory 602 and executable on the processor 601, wherein the processor 601 and the memory 602 communicate with each other via the bus 603. When the processor 601 executes the computer program, the steps of the above method are implemented, for example, including: obtaining at least one image of a target object in a space station; obtaining first positioning information of the target object in a body area network; determining target positioning information of at least one node of the target object according to at least one image, the first positioning information, and a preset first positioning model and a second positioning model; and positioning the target object in the space station according to the target positioning information.

[0069] In addition, in one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented, for example, including: obtaining at least one image of the target object in the space station; obtaining first positioning information of the target object in the body area network; determining target positioning information of at least one node of the target object based on at least one image, the first positioning information, and a preset first positioning model and a second positioning model; and positioning the target object in the space station based on the target positioning information.

[0070] In summary, in the technical solution disclosed in the present invention, the image of the target object in the space station can be fused with the body area network positioning information, thereby improving the accuracy of positioning the object in the space station.

[0071] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A spatial positioning method, comprising: Acquire at least one image of a target object within the space station; Acquire first positioning information of the target object in the body area network; Determine target positioning information of at least one node of the target object according to the at least one image, the first positioning information, and a preset first positioning model and a second positioning model; Positioning the target object in the space station according to the target positioning information; The determining the target positioning information of at least one node of the target object according to the at least one image, the first positioning information, and the preset first positioning model and second positioning model includes: fusing the at least one image to obtain a fused image; Determining second positioning information of at least one node of the target object according to the fused image and the first positioning model; Determine target positioning information of the at least one node according to the first positioning information, the second positioning information and the second positioning model; The determining, according to the first positioning information, the second positioning information, and the second positioning model, the target positioning information of the at least one node includes: Combining the first positioning information with the second positioning information to obtain third positioning information; Determine a connection diagram of the at least one node according to the third positioning information and the second positioning model; The target positioning information is determined according to the connection diagram.

2. The method according to claim 1, wherein: The fusing the at least one image to obtain a fused image comprises: Extracting feature points of the target object in the at least one image; registering the at least one image according to the feature points; The at least one registered image is fused to obtain the fused image.

3. The method according to claim 1, wherein: Determining the target positioning information according to the connection diagram includes: Determining a head region of the target object according to the connection diagram; The posture of the target object is determined according to the head region.

4. The method according to claim 1, wherein: The acquiring at least one image of the target object in the space station comprises: Acquire a set of images acquired by at least one image acquisition device installed at a preset position in the space station; Images acquired at the same time by different image acquisition devices are selected from the image set.

5. The method according to claim 1, wherein: The acquiring first positioning information of the target object in the body area network includes: Acquire a body area network signal of the target object, wherein the body area network signal is sent outward after the target object receives a baseband signal sent by a body area network positioning base station arranged in the space station; The body area network signal is analyzed to determine the first positioning information.

6. A spatial positioning device, comprising: A first acquisition unit is configured to acquire at least one image of a target object in the space station; A second acquiring unit is configured to acquire first positioning information of the target object in the body area network; a position determination unit, configured to determine target positioning information of at least one node of the target object according to the at least one image, the first positioning information, and a preset first positioning model and a second positioning model; a space positioning unit, configured to position the target object in the space station according to the target positioning information; The position determination unit is further configured to: fusing the at least one image to obtain a fused image; Determining second positioning information of at least one node of the target object according to the fused image and the first positioning model; determining target positioning information of the at least one node according to the first positioning information, the second positioning information, and the second positioning model; and The position determination unit is further configured to: Combining the first positioning information with the second positioning information to obtain third positioning information; Determine a connection diagram of the at least one node according to the third positioning information and the second positioning model; The target positioning information is determined according to the connection diagram.

7. An electronic device comprising a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the spatial positioning method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the spatial positioning method according to any one of claims 1 to 5 is implemented.

Images