Target indication method, device and system, electronic equipment and storage medium

Abstract

The invention provides a target indication method, device and system, electronic equipment and a storage medium, and relates to the technical field of communication positioning. Based on the absolute position information of the target person at the current moment and the relative position information between the target person and the target ground object, obtaining the absolute position information of the target ground object at the current moment; and based on the absolute position information of the target ground object at the current moment, generating the rendering content of the target ground object at the current moment, and superposing the rendering content of the target ground object at the current moment into the target image at the current moment for near-eye viewing of the target personnel. According to the target indication method, device and system, the electronic equipment and the storage medium provided by the invention, target indication can be performed more flexibly and more simply on the basis of not occupying both hands of an operator, and the intelligent level of work such as equipment inspection, rescue and relief work or field investigation can be improved.

Description

Technical Field

[0001] The present invention relates to the field of communication positioning technology, and in particular to a target indication method, device, system, electronic equipment and storage medium. Background Art

[0002] In equipment inspection, disaster relief or on-site investigation, traditional dedicated equipment is usually used to indicate targets on-site, or remotely based on maps.

[0003] When the above-mentioned traditional special equipment is used in the related art to perform target indication on site, the above-mentioned traditional special equipment can integrate functions such as data collection, storage and analysis into one device, thereby improving work efficiency.

[0004] However, due to the high precision requirements of the integrated components, the above-mentioned traditional special equipment is usually large in size and difficult to flexibly indicate the target. In addition, the operation of the above-mentioned traditional special equipment is usually cumbersome and requires the operator's hands to be occupied during operation, which is not conducive to the development of equipment inspection, disaster relief or on-site investigation. Summary of the invention

[0005] The present invention provides a target indication method, device, system, electronic device and storage medium, which are used to solve the defects in the prior art that target indication requires occupying both hands of an operator and is difficult to perform target indication flexibly and simply, and realizes more flexible and simple target indication without occupying both hands of the operator.

[0006] The present invention provides a target indication method, which is applied to a head-mounted display device, wherein the head-mounted display device comprises a near-eye display and an image sensor, wherein the near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor;

[0007] The method comprises:

[0008] When it is determined that the aiming mark in the target image has been aimed at the target object, obtaining target position information at the current moment, wherein the target person is a person wearing the head mounted display device, and the target position information includes absolute position information of the target person and relative position information between the target person and the target object;

[0009] Based on the target position information at the current moment, obtaining the absolute position information of the target object at the current moment;

[0010] Based on the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment is generated, and the rendering content of the target object at the current moment is superimposed on the target image at the current moment for the target person to view with close eyes.

[0011] According to a target indication method provided by the present invention, the step of obtaining target location information at the current moment includes:

[0012] Obtaining the coordinate value of the target person at the current moment in the earth coordinate system as the absolute position information of the target person at the current moment;

[0013] Acquire direction finding data of the target person at the current moment in the earth coordinate system, and determine the direction angle information of the target person relative to the target object at the current moment based on the direction finding data;

[0014] The Euler angle data of the target person's head is obtained, and based on the Euler angle data of the head, the pitch angle information and roll angle information of the target person relative to the target object are determined, wherein the Euler angle data of the head includes the direction angle information, pitch angle information and roll angle information of the target person's head.

[0015] According to a target indication method provided by the present invention, generating rendering content of the target object based on the absolute position information of the target object includes:

[0016] Based on the absolute position information of the target person at the current moment and the absolute position information at the previous moment, obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment;

[0017] When it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is less than a preset threshold, based on the Euler angle data of the head of the target person at the current moment, the three-dimensional rotation information of the target person at the current moment is obtained;

[0018] Based on the three-dimensional translation information of the target person at the current moment relative to the previous moment, the three-dimensional rotation information of the target person at the current moment and the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment is obtained.

[0019] According to a target indication method provided by the present invention, after obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment, the method further includes:

[0020] When it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is not less than the preset threshold, the absolute position information of the target object at the current moment is updated.

[0021] According to a target indication method provided by the present invention, the step of obtaining the absolute position information of the target object at the current moment based on the target position information at the current moment includes:

[0022] Based on the absolute position information of the target person at the current moment, obtaining the first position coordinates in the target coordinate system at the current moment, wherein the target coordinate system is any coordinate system except the earth coordinate system;

[0023] Based on the relative position information between the target person and the target object at the current moment, determining the coordinate correction value in the target coordinate system at the current moment;

[0024] Based on the first position coordinates in the target coordinate system at the current moment and the coordinate correction value in the target coordinate system at the current moment, obtaining the second position coordinates in the target coordinate system at the current moment;

[0025] Based on the second position coordinates in the target coordinate system at the current moment, the absolute position information of the target object at the current moment is obtained.

[0026] The present invention also provides a target indication device, which is applied to a head-mounted display device, wherein the head-mounted display device comprises a near-eye display and an image sensor, wherein the near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor;

[0027] The device comprises:

[0028] a perception module, configured to obtain target position information at a current moment when it is determined that the aiming mark in the target image has aimed at a target object, wherein the target person is a person wearing the head mounted display device, and the target position information includes absolute position information of the target person and relative position information between the target person and the target object;

[0029] A sighting module, used for acquiring the absolute position information of the target object at the current moment based on the target position information at the current moment;

[0030] A rendering module is used to generate the rendering content of the target object at the current moment based on the absolute position information of the target object at the current moment, and superimpose the rendering content of the target object at the current moment on the target image at the current moment for the target person to view with close eyes.

[0031] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the target indication methods described above is implemented.

[0032] The present invention also provides a target indication system, comprising: a head mounted display device and the electronic device as described above; the head mounted display device is electrically connected to the electronic device;

[0033] The head mounted display device comprises a near-eye display and an image sensor. The near-eye display is used to display a target image carrying a sighting mark. The target image is acquired by the image sensor.

[0034] A target indication system provided by the present invention further includes: a positioning device, an inertial navigation device and a distance measuring device; the positioning device, the inertial navigation device and the distance measuring device are arranged on the head mounted display device; the positioning device, the inertial navigation device and the distance measuring device are electrically connected to the electronic device respectively;

[0035] The positioning device is used to collect the geographical location information and direction-finding data of the target person in the earth coordinate system, and send the geographical location information and direction-finding data of the target person in the earth coordinate system to the electronic device;

[0036] The inertial navigation device is used to collect Euler angle data of the target person's head and send the Euler angle data of the target person's head to the electronic device, wherein the Euler angle data of the head includes azimuth angle information, pitch angle information and roll angle information of the target person's head;

[0037] The distance measuring device is used to collect the straight-line distance between the target person and the target object.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the target indication method described in any one of the above is implemented.

[0039] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the target indication methods described above.

[0040] The target indication method, device, system, electronic device and storage medium provided by the present invention obtain the absolute position information of the target person and the relative position information between the target person and the target object as the target position information when it is determined that the aiming mark in the target image has been aimed at the target object, and then obtain the absolute position information of the target object based on the target position information, generate the rendering content of the target object based on the absolute position information of the target object, and superimpose the rendering content of the target object into the target image for the target person to view with close eyes. The target indication can be performed more flexibly and simply without occupying the operator's hands, and the intelligence level of equipment inspection, emergency rescue or on-site investigation and other work can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of the target indication method provided by the present invention;

[0043] Figure 2 is a schematic diagram of coordinate system conversion in the target indication method provided by the present invention;

[0044] Figure 3 It is a structural schematic diagram of the target indication device provided by the present invention;

[0045] Figure 4 is a schematic diagram of the structure of an electronic device provided by the present invention;

[0046] Figure 5 It is a structural schematic diagram of the target indication system provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0049] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present invention, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include the plural forms. The terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0050] Figure 1 FIG. 1 is a flow chart of the target indication method provided by the present invention. Figure 1 As shown, the target indication method of the mixed reality technology provided by the embodiment of the present invention is applied to a head-mounted display device, and the head-mounted display device includes a near-eye display and an image sensor. The near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor. The method includes: step 101, when it is determined that the sighting mark in the target image has been aimed at the target object, the target position information at the current moment is obtained, the target person is a person wearing a head-mounted display device, and the target position information includes the absolute position information of the target person and the relative position information between the target person and the target object.

[0051] It should be noted that the execution subject of the embodiment of the present invention is a target indication device. The device can be integrated on a control chip in a head-mounted display device, and the control chip can complete independent tasks matching the use scenario of the head-mounted display device, and can also communicate with other electronic devices to complete collaborative tasks in multiple scenarios.

[0052] In the embodiment of the present invention, a person wearing the head mounted display device can be determined as a target person; and a ground feature that needs to be indicated can be determined as a target ground feature.

[0053] In the embodiment of the present invention, the image captured by the image sensor in the head mounted display device is determined as the target image.

[0054] It can be understood that the above target image changes dynamically due to the movement of the target person.

[0055] The near-eye display in the head-mounted display device can display the target image and add a sighting mark to the target image, wherein the sighting mark refers to a mark or mark used in the aiming process to assist in accurately aiming at the target.

[0056] It is understandable that the target image is dynamically changing, and the target image at the current moment may be the same as or different from the target image at the previous moment. However, the position of the aiming mark in the target image at the current moment and the position in the target image at the previous moment are fixed. Optionally, the aiming mark is always located at the geometric center of the target image.

[0057] Optionally, the head-mounted display device may be any one of augmented reality glasses, mixed reality glasses, and extended reality glasses.

[0058] The application scenarios of the target indication method provided by the embodiment of the present invention may include, when it is determined that the aiming mark in the above-mentioned target image is currently aimed at the target object, based on the target indication method provided by the present invention, applying the virtual information recording the target object to the real world, so as to achieve the matching of the real scene changes with the virtual scene changes in the mixed (augmented) reality three-dimensional scene, and by superimposing the virtual information of the target object on the target image at the current moment, thereby indicating the target object to the target personnel and guiding the target to the real geographical location where the target object is located.

[0059] Specifically, in step 101, the target person wearing the above-mentioned head-mounted display device can continuously obtain images of the surrounding environment through the image sensor in the head-mounted display device. At the same time, based on the positioning device, the ranging device, etc., the absolute position information of the target person wearing the head-mounted display device at the current moment can be obtained, and the relative position information of the target person relative to the target object at the current moment can be obtained, and the two can be integrated into the target position information at the current moment.

[0060] The relative position information of the target person relative to the target object may include the straight-line distance between the target person and the target object. The straight-line distance between the target person and the target object may be acquired based on a distance measuring device.

[0061] It should be noted that the embodiments of the present invention can determine whether the aiming mark is currently aimed at the target object in a variety of ways. For example, it can determine whether the aiming mark is currently aimed at the target object based on the position information of the aiming mark in the above-mentioned target image at the current moment and the position information of the target object in the above-mentioned target image at the current moment.

[0062] Step 102: Based on the target position information at the current moment, obtain the absolute position information of the target object at the current moment.

[0063] Specifically, if the position information of the aiming mark in the above-mentioned target image at the current moment is the same as the position information of the target object in the above-mentioned target image at the current moment, it can be determined that the aiming mark is currently aimed at the target object, that is, it means that the target person is currently facing the target object. Based on the straight-line distance between the target person and the target object at the current moment and the absolute position information of the target person at the current moment, the coordinate value of the target object in the earth coordinate system at the current moment can be calculated by numerical calculation as the absolute position information of the target object at the current moment.

[0064] It should be noted that if the current position information of the aiming mark in the above target image is different from the current position information of the target object in the above target image, it can be determined that the aiming mark is not aimed at the target object at the current moment, and the step of obtaining the target position information at the current moment will not be triggered.

[0065] Step 103: Generate the rendering content of the target object at the current moment based on the absolute position information of the target object at the current moment, and superimpose the rendering content of the target object at the current moment on the target image at the current moment for the target person to view with close eyes.

[0066] It should be noted that the rendering content of the target object changes with the translation change trend and rotation change trend that occurs when the target person of the head mounted display device approaches the absolute position information of the target object.

[0067] Specifically, in an embodiment of the present invention, based on the absolute position information of the target object at the current moment, combined with the posture changes (including translation change trends and rotation change trends) of the target person in the process of approaching the target object, a projection transformation model of the virtual image and the real image of the target object at the current moment can be solved as the rendering content of the target object at the current moment.

[0068] After obtaining the rendering content of the target object at the current moment, the rendering content of the target object at the current moment can be superimposed on the target image at the current moment and fused and rendered for close-up viewing by the target person.

[0069] The embodiment of the present invention obtains the absolute position information of the target person and the relative position information between the target person and the target object as the target position information when it is determined that the aiming mark in the target image has been aimed at the target object, and then obtains the absolute position information of the target object based on the target position information, generates rendering content of the target object based on the absolute position information of the target object, and superimposes the rendering content of the target object into the target image for the target person to view with close eyes. This can perform target indication more flexibly and simply without occupying the operator's hands, and can improve the intelligence level of equipment inspection, emergency rescue or on-site investigation and other work.

[0070] On the basis of any of the above embodiments, obtaining the target position information includes: obtaining the coordinate value of the target person in the earth coordinate system at the current moment as the absolute position information of the target person at the current moment.

[0071] Specifically, in the embodiment of the present invention, the positioning device carried by the target person can be used to obtain the coordinate value of the target person in the earth coordinate system at the current moment as the absolute position information of the target person at the current moment. The coordinate value in the earth coordinate system can be expressed as [longitude, latitude, altitude].

[0072] The direction-finding data of the target person at the current moment in the earth coordinate system is obtained, and based on the direction-finding data, the azimuth angle information of the target person relative to the target object at the current moment is determined.

[0073] Specifically, in the embodiment of the present invention, the direction finding data of the target person in the earth coordinate system at the current moment can be obtained by using the direction finding device carried by the target person.

[0074] After obtaining the direction-finding data of the target person at the current moment in the earth coordinate system, the direction angle information (Yaw) of the target person relative to the target object at the current moment can be selected or fused and solved by performing stability analysis on the direction-finding data of the target person at the current moment in the earth coordinate system.

[0075] The Euler angle data of the target person's head is obtained, and based on the Euler angle data of the head, the pitch angle information and roll angle information of the target person relative to the target object are determined. The Euler angle data of the head includes the direction angle information, pitch angle information and roll angle information of the target person's head.

[0076] Specifically, in an embodiment of the present invention, the inertial navigation device carried by the target person can be used to obtain the azimuth angle information, pitch angle information, and roll angle information formed by the rotation of the target person's head around three axes in a coordinate system established with the target person's head as the origin of the coordinate system, as the Euler angle data of the target person's head.

[0077] After obtaining the Euler angle data of the target person's head, the pitch angle information (Pitch) and roll angle information (Roll) of the target person relative to the target object can be obtained by conversion based on the Euler angle data of the head.

[0078] The embodiment of the present invention obtains the coordinate value of the target person in the earth coordinate system as the absolute position information of the target person, obtains the angular direction information of the target person relative to the target object based on the direction-finding data of the target person in the earth coordinate system, obtains the pitch angle information and roll angle information of the target person relative to the target object based on the Euler angle data of the target person's head, and can unify the relevant data of the target person and the target object in the same coordinate system, and can express the relative position relationship between the target person and the target object in the above coordinate system, which can provide more accurate data basis for the subsequent generation of rendering content of the target object.

[0079] Based on any of the above embodiments, the rendering content of the target object is generated based on the absolute position information of the target object, including: based on the absolute position information of the target person at the current moment and the absolute position information at the previous moment, obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment.

[0080] Specifically, by means of numerical calculation, the distance between the absolute position information of the target person at the current moment and the absolute position information of the target person at the previous moment can be calculated.

[0081] After obtaining the distance between the absolute position information of the target person at the current moment and the absolute position information of the target person at the previous moment, the distance can be used as the three-dimensional translation information of the target person at the current moment relative to the previous moment.

[0082] When it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is less than a preset threshold, the three-dimensional rotation information of the target person at the current moment is obtained based on the Euler angle data of the head of the target person at the current moment;

[0083] Specifically, the distance between the absolute position information of the target person at the current moment and the absolute position information of the target person at the previous moment in the three-dimensional translation information of the target person at the current moment relative to the previous moment is compared with a preset threshold.

[0084] If the above distance is less than the preset threshold, it can be explained that the target person has a small displacement between the current moment and the previous moment, and then the absolute position information of the target object at the current moment can be determined to be valid, and then the Euler angle data of the target person's head at the current moment can be determined as the three-dimensional rotation information of the target person at the current moment.

[0085] Based on the three-dimensional translation information of the target person at the current moment relative to the previous moment, the three-dimensional rotation information of the target person at the current moment and the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment is obtained.

[0086] Specifically, based on the absolute position information of the target object at the current moment, combined with the three-dimensional translation information of the target person at the current moment relative to the previous moment and the three-dimensional rotation information of the target person at the current moment, the rendering content of the target object at the current moment can be obtained, whose scale is consistent with the real world and matches the head position and posture of the target person.

[0087] The embodiment of the present invention can sense the 3D rotation of the target person by using an external device and compensate for the delay when the target person's head moves or rotates quickly, thereby greatly reducing the dizziness caused by display lag to the target person.

[0088] Based on any of the above embodiments, after obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment, the method also includes: when it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is not less than a preset threshold, updating the absolute position information of the target object at the current moment.

[0089] Specifically, the distance between the absolute position information of the target person at the current moment and the absolute position information of the target person at the previous moment in the three-dimensional translation information of the target person at the current moment relative to the previous moment is compared with a preset threshold.

[0090] If the above distance is greater than or equal to the preset threshold, it can be explained that the target person has a large displacement between the current moment and the previous moment, and it can be considered that the absolute position information of the target object at the current moment is invalid. It is necessary to recalculate the absolute position information of the target object at the current moment, and based on the solution result, update the absolute position information of the target object at the current moment.

[0091] The embodiment of the present invention can ensure the timeliness of scene rendering by updating the absolute position information of the target object at the current moment when the three-dimensional translation information of the target person at the current moment relative to the previous moment is not less than the preset threshold.

[0092] On the basis of any of the above embodiments, the step of obtaining the absolute position information of the target object at the current moment based on the target position information at the current moment includes: obtaining the first position coordinates in the target coordinate system at the current moment based on the absolute position information of the target person at the current moment, wherein the target coordinate system is any coordinate system other than the earth coordinate system;

[0093] Specifically, through coordinate system conversion, the coordinates of the target person at the current moment in the earth coordinate system can be converted into the first position coordinates in the target coordinate system at the current moment (such as Cartesian coordinate system, Gaussian plane rectangular coordinate system or space rectangular coordinate system).

[0094] Figure 2Schematic diagram of coordinate system conversion in the target indication method provided by the present invention. Figure 2 As shown in FIG. 1 , taking the coordinates and direction angles measured by Beidou or GPS as an example, if i represents the target person at the current moment, j represents the target object at the current moment, and the absolute position information of the target person at the current moment includes the geodetic longitude L, the geodetic latitude B, and the geodetic height H, the absolute position information of the target person at the current moment can be converted into the first position coordinate in the target coordinate system at the current moment through the coordinate system conversion. The first position coordinate in the target coordinate system at the current moment can be expressed as (X i ,Y i ,Z i ).

[0095] Based on the relative position information between the target person and the target object at the current moment, a coordinate correction value in the target coordinate system at the current moment is determined.

[0096] Specifically, based on the absolute position information of the target person at the current moment, the direction angle α of the target person's head at the current moment ij and the pitch angle β ij As well as the straight-line distance D between the target object and the target person at the current moment, the three-axis coordinate increment (ΔX ij ,ΔY ij ,ΔZ ij ), the specific calculation formula is as follows:

[0097] ΔX ij =D·cosβ ij cosα ij

[0098] ΔY ij =D·cosβ ij sinα ij

[0099] ΔZ ij =D·sinβ ij

[0100] Calculate the three-axis coordinate increment (ΔX ij ,ΔY ij ,ΔZ ij ) can be used to calculate the three-axis coordinate increment (ΔX ij ,ΔY ij ,ΔZ ij ) is determined as the coordinate value correction value in the target coordinate system at the current moment.

[0101] Based on the first position coordinates in the target coordinate system at the current moment and the coordinate correction value in the target coordinate system at the current moment, the second position coordinates in the target coordinate system at the current moment are obtained.

[0102] Specifically, after obtaining the coordinate value correction value at the current moment, the coordinate value correction value at the current moment may be added to the first position coordinate in the target coordinate system at the current moment, to obtain the second position coordinate in the target coordinate system at the current moment.

[0103] The second position coordinate (X′) in the target coordinate system at the current moment i ,Y′ i ,Z′ i ) can be calculated using the following formula:

[0104] X i ′=X i +ΔX ij

[0105] Y i ′=Y i +ΔY ij

[0106] Z i ′=Z i +ΔZ ij

[0107] Based on the second position coordinates in the target coordinate system at the current moment, the absolute position information of the target object at the current moment is obtained.

[0108] Specifically, through coordinate system conversion, the second position coordinates in the target coordinate system at the current moment can be converted into coordinate values ​​in the earth coordinate system at the current moment, and then the above coordinate values ​​can be used as the absolute position information of the target object at the current moment.

[0109] The embodiment of the present invention can more accurately calculate the absolute position information of the target object at the current moment based on the absolute position information of the target person at the current moment and the relative position information between the target person and the target object at the current moment, thereby improving the accuracy of target indication.

[0110] Figure 3 Schematic diagram of the structure of the target indication device provided by the present invention. Figure 3 The target indication device provided by the present invention is described. The target indication device described below and the target indication method provided by the present invention described above can be referred to each other. The target indication device provided by the present invention is applied to a head-mounted display device, and the head-mounted display device includes a near-eye display and an image sensor. The near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor. Figure 3 As shown, the device includes: a perception module 301, a sighting module 302 and a rendering module 303, wherein:

[0111] The perception module 301 is used to obtain the target position information at the current moment when it is determined that the aiming mark in the target image has aimed at the target object, the target person is a person wearing the head mounted display device, and the target position information includes the absolute position information of the target person and the relative position information between the target person and the target object;

[0112] The aiming module 302 is used to obtain the absolute position information of the target object at the current moment based on the target position information at the current moment;

[0113] The rendering module 303 is used to generate the rendering content of the target object at the current moment based on the absolute position information of the target object at the current moment, and superimpose the rendering content of the target object at the current moment on the target image at the current moment for the target person to view with close eyes.

[0114] Specifically, the perception module 301, the aiming module 302 and the rendering module 303 are electrically connected.

[0115] The target indication device provided in the embodiment of the present invention is used to execute the above-mentioned target indication method of the present invention. Its implementation method is consistent with the implementation method of the target indication method provided by the present invention and can achieve the same beneficial effects, which will not be repeated here.

[0116] The target indication device in the embodiment of the present invention obtains the absolute position information of the target person and the relative position information between the target person and the target object as the target position information when it is determined that the aiming mark in the target image has been aimed at the target object, and then obtains the absolute position information of the target object based on the target position information, generates the rendering content of the target object based on the absolute position information of the target object, and superimposes the rendering content of the target object on the target image for the target person to view with close eyes. It can perform target indication more flexibly and simply without occupying the operator's hands, and can improve the intelligence level of equipment inspection, emergency rescue or on-site investigation and other work.

[0117] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device 400 may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the target indication method, which is applied to a head-mounted display device, and the head-mounted display device includes a near-eye display and an image sensor. The near-eye display is used to display a target image carrying an aiming mark, and the target image is acquired by the image sensor; the method includes: when it is determined that the aiming mark in the target image has been aimed at the target object, obtaining the target position information at the current moment, the target person is a person wearing a head-mounted display device, and the target position information includes the absolute position information of the target person and the relative position information between the target person and the target object; based on the target position information at the current moment, obtaining the absolute position information of the target object at the current moment; based on the absolute position information of the target object at the current moment, generating the rendering content of the target object at the current moment, and superimposing the rendering content of the target object at the current moment on the target image at the current moment for the target person to view with near eye.

[0118] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0119] Figure 5 Schematic diagram of the target indication system provided by the present invention. Figure 5 As shown, based on any of the above embodiments, a target indication system provided by an embodiment of the present invention includes a head mounted display device 501 and an electronic device 400; the head mounted display device 501 is electrically connected to the electronic device 400;

[0120] The head mounted display device 501 includes a near-eye display and an image sensor. The near-eye display is used to display a target image carrying a sighting mark. The target image is acquired by the image sensor.

[0121] The head mounted display device 501 may be any one of augmented reality glasses, mixed reality glasses and extended reality glasses.

[0122] Specifically, the target indication system is composed of a head-mounted display device 501 that can be used as protective gear and an electronic device 400 with independent computing functions.

[0123] It should be noted that the electronic device 400 in the embodiment of the present invention is used to execute the above-mentioned target indication method of the present invention, and its implementation method is consistent with the implementation method of the target indication method provided by the present invention, and can achieve the same beneficial effects, which will not be repeated here.

[0124] The target indication system in the embodiment of the present invention includes an electronic device and a head-mounted display device. The electronic device obtains the absolute position information of the target person and the relative position information between the target person and the target object as the target position information when determining that the aiming mark in the target image has been aimed at the target object, and then obtains the absolute position information of the target object based on the target position information, generates rendering content of the target object based on the absolute position information of the target object, and superimposes the rendering content of the target object on the target image for the target person to view with close eyes. The target indication can be performed more flexibly and simply without occupying the operator's hands, and the intelligence level of equipment inspection, emergency rescue or on-site investigation and other work can be improved.

[0125] On the basis of any of the above embodiments, the target indication system provided by the embodiment of the present invention further includes: a positioning device, an inertial navigation device and a distance measuring device;

[0126] The positioning device, the inertial navigation device and the distance measuring device are arranged on the head mounted display device; the positioning device, the inertial navigation device and the distance measuring device are electrically connected to the electronic device respectively;

[0127] The positioning device is used to collect the geographical location information and direction-finding data of the target person in the earth coordinate system, and send the geographical location information and direction-finding data of the target person in the earth coordinate system to the electronic device;

[0128] The inertial navigation device is used to collect Euler angle data of the target person's head and send the Euler angle data of the target person's head to the electronic device, wherein the Euler angle data of the head includes azimuth angle information, pitch angle information and roll angle information of the target person's head;

[0129] The distance measuring device is used to collect the straight-line distance between the target person and the target object.

[0130] Specifically, the positioning device includes but is not limited to Beidou, Global Positioning System (GPS), GLONASS, and Galileo satellite navigation modules, which can perform real-time positioning and navigation on a global scale. It can not only measure the longitude and latitude and altitude of the point (i.e., geographic location information), but also additionally set the geomagnetic direction (i.e., direction finding data) measured by the Global Navigation Satellite System (GNSS) antenna.

[0131] The inertial navigation device includes but is not limited to an accelerometer, a gyroscope, etc., to measure three-axis angular velocity, acceleration, and magnetic declination.

[0132] The distance measuring device includes but is not limited to laser, radar, and ultrasonic rangefinder to measure the distance of the target object relative to the target person, and use multiple measurements within a certain period of time to calculate the average, median, sliding window, weighted average, and filtering methods to process the measurement data.

[0133] The embodiments of the present invention can integrate various sensing devices on a head-mounted display device, so that a target person wearing the head-mounted display device can aim at the target object through the integrated head-mounted display device for target indication, so that the observation content of the target indication can be intuitively presented in front of the target person and is always consistent with his or her own actions, freeing the target person's hands, thereby improving the flexibility of the target indication work.

[0134] In addition, the logic instructions can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0135] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target indication method provided by the above methods, which is applied to a head-mounted display device. The head-mounted display device includes a near-eye display and an image sensor. The near-eye display is used to display a target image carrying an aiming mark, and the target image is acquired by the image sensor; the method includes: when it is determined that the aiming mark in the target image has been aimed at the target object, obtaining the target position information at the current moment, the target person is a person wearing a head-mounted display device, and the target position information includes the absolute position information of the target person and the relative position information between the target person and the target object; based on the target position information at the current moment, obtaining the absolute position information of the target object at the current moment; based on the absolute position information of the target object at the current moment, generating the rendering content of the target object at the current moment, and superimposing the rendering content of the target object at the current moment on the target image at the current moment, so that the target person can view it with near eye.

[0136] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the target indication method provided by the above-mentioned methods, and is applied to a head-mounted display device, the head-mounted display device including a near-eye display and an image sensor, the near-eye display being used to display a target image carrying an aiming mark, the target image being acquired by the image sensor; the method comprising: upon determining that the aiming mark in the target image has been aimed at the target object, obtaining target position information at a current moment, the target person being a person wearing the head-mounted display device, the target position information comprising the absolute position information of the target person and the relative position information between the target person and the target object; based on the target position information at a current moment, obtaining the absolute position information of the target object at a current moment; based on the absolute position information of the target object at a current moment, generating the rendering content of the target object at a current moment, and superimposing the rendering content of the target object at a current moment onto the target image at a current moment, for the target person to view with near eye.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target indication method, characterized in that: Applied to a head-mounted display device, the head-mounted display device includes a near-eye display and an image sensor, the near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor; The method comprises: When it is determined that the aiming mark in the target image has been aimed at the target object, obtaining target position information at the current moment, where the target person is a person wearing the head mounted display device, and the target position information includes absolute position information of the target person and relative position information between the target person and the target object; Based on the target position information at the current moment, obtaining the absolute position information of the target object at the current moment; Based on the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment is generated, and the rendering content of the target object at the current moment is superimposed on the target image at the current moment for the target person to view with close eyes.

2. The target indication method according to claim 1, characterized in that: The obtaining of the target location information at the current moment includes: Obtaining the coordinate value of the target person at the current moment in the earth coordinate system as the absolute position information of the target person at the current moment; Acquire direction finding data of the target person at the current moment in the earth coordinate system, and determine the direction angle information of the target person relative to the target object at the current moment based on the direction finding data; The Euler angle data of the target person's head is obtained, and based on the Euler angle data of the head, the pitch angle information and roll angle information of the target person relative to the target object are determined, wherein the Euler angle data of the head includes the direction angle information, pitch angle information and roll angle information of the target person's head.

3. The target indication method according to claim 2, characterized in that: The generating, based on the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment comprises: Based on the absolute position information of the target person at the current moment and the absolute position information at the previous moment, obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment; When it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is less than a preset threshold, based on the Euler angle data of the head of the target person at the current moment, the three-dimensional rotation information of the target person at the current moment is obtained; Based on the three-dimensional translation information of the target person at the current moment relative to the previous moment, the three-dimensional rotation information of the target person at the current moment and the absolute position information of the target object at the current moment, the rendering content of the target object at the current moment is obtained.

4. The target indication method according to claim 3, characterized in that: After obtaining the three-dimensional translation information of the target person at the current moment relative to the previous moment, the method further includes: When it is determined that the three-dimensional translation information of the target person at the current moment relative to the previous moment is not less than the preset threshold, the absolute position information of the target object at the current moment is updated.

5. The target indication method according to claim 1, characterized in that: The step of acquiring the absolute position information of the target object at the current moment based on the target position information at the current moment includes: Based on the absolute position information of the target person at the current moment, obtaining the first position coordinates in the target coordinate system at the current moment, wherein the target coordinate system is any coordinate system except the earth coordinate system; Based on the relative position information between the target person and the target object at the current moment, determining the coordinate correction value in the target coordinate system at the current moment; Based on the first position coordinates in the target coordinate system at the current moment and the coordinate correction value in the target coordinate system at the current moment, obtaining the second position coordinates in the target coordinate system at the current moment; Based on the second position coordinates in the target coordinate system at the current moment, the absolute position information of the target object at the current moment is obtained.

6. A target indicating device, characterized in that: Applied to a head-mounted display device, the head-mounted display device includes a near-eye display and an image sensor, the near-eye display is used to display a target image carrying a sighting mark, and the target image is acquired by the image sensor; The device comprises: A perception module, configured to obtain target position information at a current moment when it is determined that the aiming mark in the target image has aimed at a target object, wherein the target person is a person wearing the head mounted display device, and the target position information includes absolute position information of the target person and relative position information between the target person and the target object; A sighting module, used for acquiring the absolute position information of the target object at the current moment based on the target position information at the current moment; A rendering module is used to generate the rendering content of the target object at the current moment based on the absolute position information of the target object at the current moment, and superimpose the rendering content of the target object at the current moment on the target image at the current moment for the target person to view with close eyes.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the target indication method according to any one of claims 1 to 5 are implemented.

8. A target indication system, characterized in that: include: A head mounted display device and an electronic device as claimed in claim 7; The head mounted display device is electrically connected to the electronic device; The head mounted display device comprises a near-eye display and an image sensor. The near-eye display is used to display a target image carrying a sighting mark. The target image is acquired by the image sensor.

9. The target indication system according to claim 8, characterized in that: Also includes: A positioning device, an inertial navigation device and a distance measuring device; the positioning device, the inertial navigation device and the distance measuring device are arranged on the head mounted display device; The positioning device, the inertial navigation device and the distance measuring device are electrically connected to the electronic equipment respectively; The positioning device is used to collect the geographical location information and direction-finding data of the target person in the earth coordinate system, and send the geographical location information and direction-finding data of the target person in the earth coordinate system to the electronic device; The inertial navigation device is used to collect Euler angle data of the target person's head and send the Euler angle data of the target person's head to the electronic device, wherein the Euler angle data of the head includes azimuth angle information, pitch angle information and roll angle information of the target person's head; The distance measuring device is used to collect the straight-line distance between the target person and the target object.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target indication method according to any one of claims 1 to 5 is implemented.

Images