Target sharing method based on AR glasses and AR glasses

Abstract

The invention relates to a target sharing method based on AR glasses and the AR glasses. The target sharing method based on the AR glasses comprises the steps that first AR glasses of a first user acquire target feature information of a target, a first azimuth coordinate of the first user and a first relative space coordinate of the target relative to the first user; sending target information and a first azimuth coordinate of the first user to second AR glasses of at least one second user, wherein the target information at least comprises feature information of a target and a first relative space coordinate; the second AR glasses calculate and obtain a second relative space coordinate of the target relative to the second user; judging whether the second relative space coordinate is located in the display field of view of the second AR glasses or not; and if the second relative space coordinate is located in the display visual field range of the second AR glasses, marking the target on a virtual display panel of the second AR glasses, otherwise, marking prompt information of the target on the virtual display panel of the second AR glasses.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of augmented reality, in particular to an AR glasses based target sharing method and AR glasses. BACKGROUND

[0002] In the field of multi-person collaborative work, such as military operations, emergency rescue, etc., especially in application scenarios that require silence or high noise, whether the position information of a specific target can be efficiently and reliably shared between collaborative users becomes a key factor affecting the efficiency of collaborative work. SUMMARY

[0003] The purpose of the present disclosure is to provide an AR glasses based target sharing method and AR glasses to at least partially overcome the problems existing in the related art.

[0004] To achieve the above-mentioned purpose, the first aspect of the present disclosure provides an AR glasses based target sharing method, which comprises:

[0005] The first AR glasses of a first user obtain target feature information of a target, a first position coordinate of the first user and a first relative spatial coordinate of the target relative to the first user; the target information and the first position coordinate of the first user are sent to second AR glasses of at least one second user, wherein the target information at least includes the feature information of the target and the first relative spatial coordinate; the second AR glasses calculate a second relative spatial coordinate of the target relative to the second user; it is judged whether the second relative spatial coordinate is located within a display field of view range of the second AR glasses; if the second relative spatial coordinate is located within the display field of view range of the second AR glasses, the target is marked on a virtual display panel of the second AR glasses, otherwise, prompt information of the target is marked on the virtual display panel of the second AR glasses.

[0006] Optionally, according to the second relative spatial coordinate, a display position of the target on the virtual display panel is calculated; a recognition feature of the target is presented at the display position, wherein the presentation form of the recognition feature is adaptively scaled according to distance information in the second relative spatial coordinate.

[0007] Optionally, the second AR glasses collect a current environment image; a local image feature corresponding to the second relative spatial coordinate is extracted from the environment image; the local image feature is compared with a recognition feature in the received target feature information in terms of similarity; if the similarity is higher than a preset threshold, the target is marked in a first display mode; if the similarity is lower than the preset threshold, the target is marked in a second display mode.

[0008] Optionally, the first display mode and the second display mode are distinguished by one or more of a combination of color, contour style, transparency, or blinking frequency.

[0009] Optionally, the prompt information includes an offset angle and / or a straight-line distance of the target relative to a current center of field of view of the second user.

[0010] Optionally, the first AR glasses continuously update data information, the data information at least including the first position coordinate; and the updated data information is sent to the second AR glasses.

[0011] According to a second aspect of the present disclosure, an AR glasses is also provided, which comprises an image acquisition module, a position acquisition module, an augmented display module, a communication module and a master control module. The image acquisition module is configured to acquire environmental images around a user. The position acquisition module is configured to acquire a position coordinate of the user. The augmented display module is configured to provide a virtual display panel. The communication module is configured to perform data communication. The master control module is electrically connected with the image acquisition module, the position acquisition module, the augmented display module and the communication module, and is configured to execute the steps of the above method.

[0012] Optionally, the position acquisition module comprises an inertial measurement unit, a global satellite positioning system receiver and a visual simultaneous localization and mapping processor.

[0013] Through the above technical solution, the first AR glasses acquires target information and a first position coordinate and sends them to a second user. The second AR glasses of the second user converts the target position to a coordinate system with the second user as the origin according to the received data information, judges whether the target is within the field of view of the second user, and triggers two different display modes according to the judgment, i.e. directly marking if the target is within the field of view, and providing a guiding prompt if the target is outside the field of view. In this way, cross-user visual spatial information sharing can be realized, blind searching outside the field of view can be avoided, and quick locking within the field of view can be ensured, which significantly improves the efficiency of target positioning. Moreover, the whole process does not require voice communication, and is especially suitable for reconnaissance, rescue and other communication scenarios that require silence or voice communication is inconvenient, which can improve the situation awareness and execution efficiency of multi-person collaborative tasks.

[0014] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, which together with the following detailed description, serve to explain the present disclosure. In the drawings:

[0016] Figure 1is a flowchart of a method for information sharing based on AR glasses provided by an embodiment of the present disclosure.

[0017] Figure 2 is a schematic diagram of target annotation on a virtual display panel when a target falls within an augmented display field of view range, provided by an embodiment of the present disclosure.

[0018] Figure 3 is a schematic diagram of prompt information on a virtual display panel when a target does not fall within an augmented display field of view range, provided by an embodiment of the present disclosure.

[0019] Figure 4 is a schematic diagram of a target not falling within an augmented display field of view range, provided by an embodiment of the present disclosure.

[0020] Figure 5 is a schematic diagram of a module block diagram of AR glasses, provided by an embodiment of the present disclosure.

[0021] Legend of Reference Signs

[0022] 10-AR glasses; 1-image acquisition module; 2-position acquisition module; 3-augmented display module; 4-communication module; 5-master control module; 101-center line of field of view; 102-virtual display panel; 200-target; 201-virtual identifier. DETAILED DESCRIPTION

[0023] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to illustrate and explain the present disclosure, and are not intended to limit the present disclosure.

[0024] It is necessary to understand in the present disclosure that the terms such as "first", "second" and the like used in the present disclosure are used to distinguish one element from another element, and do not have sequentiality and importance. In addition, in the description with reference to the drawings, the same reference signs in different drawings represent the same elements.

[0025] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0026] The method for information sharing based on AR glasses provided by the present disclosure can be applied to the application scenario of multi-user collaborative activities, and is especially suitable for application scenarios such as site reconnaissance, rescue search and the like which need to maintain silence or are inconvenient for voice communication. In the scenario of multi-user collaborative activities sharing information, multiple users respectively wear AR glasses.

[0027] As Figure 5As shown, the AR glasses 10 provided in this disclosure include an image acquisition module 1, a location acquisition module 2, an augmented display module 3, a communication module 4, and a main control module 5. The image acquisition module 1 is used to acquire environmental images around the user, the location acquisition module 2 is used to acquire the user's orientation coordinates, the augmented display module 3 is used to provide a virtual display panel 102, the communication module 4 is used for data communication, and the main control module 5 is electrically connected to the image acquisition module 1, the location acquisition module 2, the augmented display module 3, and the communication module 4.

[0028] Specifically, the image acquisition module 1 may include one or more camera components for acquiring environmental images (such as RGB images, depth images, heatmap images, etc.) around the user. The acquired environmental images are used for target 200 recognition, feature extraction, or occlusion judgment as mentioned below.

[0029] Location acquisition module 2 is used to acquire the user's orientation coordinates (such as the user's six-dimensional coordinates in a coordinate system with the Earth as the origin). Location acquisition module 2 may include an inertial measurement unit (IMU) and a Global Positioning System (GPS) receiver. For example, the IMU can provide high-frequency orientation and acceleration data, while GPS can provide the user's absolute position reference. The user's orientation coordinates can be calculated using a multi-sensor fusion positioning algorithm.

[0030] The enhanced display module 3 includes optical display modules (such as optical waveguide display modules, BirdBath display modules, prism display modules, etc.) for projecting virtual display panels 102 onto the user's retina to overlay digital information (such as images and text) with real-world scenes.

[0031] The communication module 4 is used to establish a communication connection between AR glasses 10 of different users and transmit data such as target 200 information and user location coordinates. The communication module 4 can transmit data through wireless communication methods such as 5G and Wi-Fi.

[0032] The main control module 5 is electrically connected to the image acquisition module 1, the position acquisition module 2, the enhanced display module 3, and the communication module 4 to coordinate the operation of each module.

[0033] In multi-user collaborative application scenarios, for example, a first user wears a first AR glasses and a second user wears a second AR glasses. The first user sends image information to the second user's second AR glasses through the first AR glasses, so that the second user can share the information sent by the first user through the second AR glasses.

[0034] like Figure 1 As shown, the information sharing method based on AR glasses provided in this disclosure includes the following steps:

[0035] Step S101: Obtain the target 200 feature information, the first orientation coordinates of the first user, and the first relative spatial coordinates of the target 200 relative to the first user through the first user's first AR glasses.

[0036] The feature information of target 200 may include the outline of target 200 or the thermal image of target 200 (such as the thermal imaging acquisition component included in the image acquisition module 1 of the first AR glasses).

[0037] The first relative spatial coordinates are specifically the coordinates of target 200 in a coordinate system with the first user as the origin. They can be calculated using the first user's first orientation coordinates and the image depth information containing the environmental image of target 200.

[0038] The image depth information containing the environmental image of target 200 can be obtained through image information acquired by, for example, a binocular RGB camera or a TOF camera in image acquisition module 1.

[0039] The first directional coordinates can be obtained through the position acquisition module 2 in the first AR glasses. The first directional coordinates include the position coordinates and orientation coordinates of the first user. The position coordinates are denoted as (Xa, Ya, Za), the orientation coordinates are denoted as (Yawa, Pitcha, Rolla), and the first directional coordinates are denoted as (Xa, Ya, Za, Yawa, Pitcha, Rolla).

[0040] Specifically, the location acquisition module 2 includes GPS and an inertial measurement unit (IMU). The GPS can acquire the location coordinates of the first user in a coordinate system with the Earth as the origin. The inertial measurement unit includes a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer. The orientation coordinates of the first user's first AR glasses can be calculated through a sensor fusion algorithm (such as a complementary filter or a Kalman filter).

[0041] The image depth information containing the environmental image of the target 200, acquired by the image acquisition module 1 of the first AR glasses, combined with the intrinsic parameters of the camera and the first azimuth coordinates, is used to calculate the three-dimensional offset of the target 200 in a coordinate system with the first user as the origin, using inverse operations such as perspective projection. The three-dimensional offset is denoted as (ΔXa, ΔYa, ΔZa), which is the first relative spatial coordinate. Specifically, this process involves identifying the two-dimensional pixel position of the target 200 in the image and introducing depth information (e.g., through stereo vision, comparison with a target 200 of known size, or monocular depth estimation model) to transform the two-dimensional pixel coordinates into a three-dimensional coordinate system with the first user as the origin, thereby determining the direction and distance of the target 200 relative to the user.

[0042] Step S102: Send the target 200 information and the first location coordinates of the first user to the second AR glasses of at least one second user, wherein the target 200 information includes at least the feature information of the target 200 and the first relative spatial coordinates.

[0043] There can be one or more second users. When the first user needs to share target 200 information with multiple second users, the target 200 information and the first user's first location coordinates can be sent to the corresponding second users respectively.

[0044] Step S103: The second AR glasses calculate and obtain the second relative spatial coordinates of the target 200 relative to the second user.

[0045] The second relative spatial coordinates can specifically be the coordinates of target 200 in a coordinate system with the second user as the origin. Based on the first user's first orientation coordinates, the feature information of target 200, and the second user's second orientation coordinates, the second AR glasses calculate the second relative spatial coordinates of target 200 relative to the second user through spatial coordinate transformation.

[0046] Similarly, the second user's second location coordinates can be obtained through the position acquisition module 2 of the second AR glasses, just as the first user's first location coordinates were obtained. Further details will not be provided here.

[0047] Step S104: Determine whether the second relative spatial coordinates are within the display field of view of the second AR glasses.

[0048] The second relative spatial coordinates of target 200 are transformed into pixel coordinates on the virtual display panel 102 of the second AR glasses through perspective projection transformation in the coordinate system with the second user as the origin. If the transformed pixel coordinates are within the physical pixel boundary of the virtual display panel 102, they are determined to be within the enhanced display field of view; otherwise, they are determined to be outside the enhanced display field of view.

[0049] Among them, reference Figure 3 The display field of view of the augmented display module 3 refers to the area of ​​the real world that the virtual image projected by the augmented display module 3 of the AR glasses can cover, and that the user can observe through the augmented display module 3. The augmented display field of view can be a finite conical viewing angle range determined by the optical system of the augmented display module 3, such as... Figure 3 The display field of view shown is (α,β), where α is the horizontal field of view and β is the vertical field of view.

[0050] Step S105: If the second relative spatial coordinates are within the display field of view of the second AR glasses, mark target 200 on the virtual display panel 102 of the second AR glasses; otherwise, mark target 200 with a prompt message on the virtual display panel 102 of the second AR glasses.

[0051] If the calculated second relative spatial coordinates fall within the display field of view of the second AR glasses, then the target 200 is enhanced and marked on the virtual display panel 102 of the second user. Specifically, the system calculates the display position of the target 200 on the virtual display panel 102 based on the second relative spatial coordinates, and presents a virtual identifier 201 (e.g., a highlighted outline, icon, or 3D arrow) associated with the feature information of the target 200 at this position. Figure 2 As shown, if the target 200 falls within the display field of view, the virtual identifier 201 will maintain visual spatial alignment with the target 200 in the real world, thereby enabling the user to intuitively and quickly identify and lock onto the specific target 200 they want to observe, achieving efficient visual guidance.

[0052] If the second relative spatial coordinates are outside the enhanced display field of view of the second AR glasses, a prompt message indicating the location of the target 200 is displayed on the virtual display panel 102 of the second user. Specifically, on the virtual display panel 102, for example, in the edge area of ​​the virtual display panel 102 (such as the side of the screen corresponding to the direction of deviation of the target 200), a directional guide mark (such as an arrow or fan-shaped pointer) is presented, supplemented with distance information. The dynamic pointing of this mark can accurately reflect the deviation angle between the target 200 and the center axis of the second user's field of view, thereby guiding the user to eventually bring the target 200 into the actual field of view containing the target 200 by turning their head or body, which helps the user locate the target 200.

[0053] Through the above technical solution, the first AR glasses acquire the target 200 information and its first azimuth coordinates and send them to the second user. The second user's second AR glasses, based on the received data, transform the target 200's position into a coordinate system with the second user as the origin. By determining whether the target 200 is within the second user's field of vision, two different display modes are triggered: if within the field of vision, it is directly marked; if outside the field of vision, guiding prompts are provided. This enables cross-user, intuitive, and accurate sharing of visual spatial information, avoiding blind searching outside the field of vision and ensuring rapid locking within the field of vision, significantly improving the efficiency of target 200 positioning. Furthermore, the entire process requires no voice communication, making it particularly suitable for communication scenarios such as reconnaissance and rescue where silence is required or voice communication is inconvenient, thus improving situational awareness and execution efficiency in multi-person collaborative tasks.

[0054] In some embodiments, the method of marking target 200 on the virtual display panel 102 of the second AR glasses includes:

[0055] The display position of target 200 on the virtual display panel 102 is calculated based on the second relative spatial coordinates.

[0056] The identification features of target 200 are presented at the display location, wherein the presentation form of the identification features is adaptively scaled according to distance information in the second relative spatial coordinates.

[0057] The second relative spatial coordinates are transformed by perspective projection to obtain the corresponding pixel coordinates on the virtual display panel 102 of the second user. The received target 200 identification features (such as outline) are presented at these pixel coordinates, and the target 200 is dynamically scaled according to the distance information so that the size of the displayed target 200 icon changes with the actual distance between the target 200 and the user.

[0058] The distance scaling icon conforms to the perspective rules of human vision, making the virtual markers blend more naturally with the real world and reducing cognitive dissonance for users. Users can intuitively judge the distance of target 200 by the size of the icon, obtaining more direct depth information than just numerical distance. This avoids the problems of distant target 200 markers being too large and obstructing the view, or close target 200 markers being too small to be noticed, improving display clarity and usability.

[0059] In some embodiments, the method of marking the target 200 on the virtual display panel 102 of the second AR glasses further includes:

[0060] The current environmental image is captured using a second AR glasses.

[0061] Extract local image features corresponding to the second relative spatial coordinates from environmental images.

[0062] The similarity of local image features with the identified features in the received target 200 feature information is compared.

[0063] If the similarity is higher than a preset threshold, the target 200 is labeled in the first display mode.

[0064] If the similarity is lower than a preset threshold, the target 200 is labeled in the second display mode.

[0065] After receiving the coordinates of target 200, the second AR glasses not only annotate target 200, but also extract image features of the theoretical position of target 200 by collecting the current field of view image, and perform real-time similarity comparison with the identification features of target 200 sent by the first user. Depending on whether the comparison result is higher or lower than a preset threshold, the first display mode (such as a bright solid line) or the second display mode (such as a semi-transparent dashed line) is used for annotation, thereby prompting the user whether target 200 is currently occluded or directly visible.

[0066] By introducing image recognition and comparison mechanisms, the system can distinguish the consistency between virtual coordinates and the real visual environment, providing users with the key information of "target visibility", which can effectively avoid misleading labeling caused by the presence of occlusions.

[0067] In some embodiments, the first display mode and the second display mode can be distinguished by one or more combinations of color, outline style, transparency, or flicker frequency.

[0068] The first and second display modes are distinguished by using one or more combinations of color, outline style, transparency, or blink frequency. For example, the first display mode (visible) can be outlined with a solid red line, while the second display mode (potentially obscured) can be outlined with a semi-transparent blue dashed line that blinks slowly. This allows users to quickly understand the state of target 200 and also allows for customization based on different application scenarios and user preferences, enhancing the system's adaptability and user experience.

[0069] In some embodiments, the prompting information includes the deviation angle of the target 200 relative to the second user's current field of vision center line 101, and / or, an indication of the straight-line distance between the target 200 and the second user. Thus, when the target 200 is not within the second user's enhanced display field of vision, presenting the second user with prompting information such as the deviation angle and / or straight-line distance, for example in the form of graphics, arrows, text, or symbols, can efficiently guide the second user to turn or move in the correct direction, quickly bringing the target 200 into their field of vision.

[0070] For example, such as Figure 4As shown in the diagram, the triangle symbol P represents target 200, the dot symbol O represents the current position of the second user, and the dashed line S1 starting from O indicates the direction of the second user's visual field center line 101. A dashed directional arrow is positioned between O and P. Next to the dashed directional arrow S2, the text label "50m" (i.e., the distance L between target 200 and the second user) is displayed. Between S2 and S1, the text label "10°" is displayed, representing the deviation angle Δγ of target 200 from the second user. The second user only needs to turn their head according to the guidance of the dashed directional arrow S2 to bring target 200 into their field of vision.

[0071] In addition, prompts can be displayed near the edge of the virtual display panel 102 for the second user, which can guide the user's view of the real world without excessively encroaching on the second user's central field of vision.

[0072] In some embodiments, the method further includes:

[0073] The first AR glasses continuously update data information, including target 200 information and first location coordinates, and send the updated data information to the second AR glasses.

[0074] Specifically, for example, when target 200 is in motion, after the first user locks onto target 200, the main control module 5 of the first AR glasses will continuously input data from the image acquisition module 1 and the location acquisition module 2, and calculate the first relative spatial coordinates of target 200 relative to the first user in real time using a target 200 tracking algorithm (such as KCF or SIFT feature tracking). The updated first relative spatial coordinates will then be continuously (e.g., periodically at 10-second intervals) sent to the second user's second AR glasses via the communication module 4 of the first AR glasses. This ensures that the target 200 location information shared between the first and second users remains synchronized with the dynamic changes of target 200 in the real world.

[0075] For example, when both the first user and the target 200 are in motion, after the first user locks onto the target 200, the main control module 5 of the first AR glasses will continuously receive data input from the image acquisition module 1 and the location acquisition module 2, calculate and obtain real-time information about the target 200 and the first azimuth coordinates, and continuously send them to the second user's second AR glasses.

[0076] The main control module 5 of the AR glasses 10 provided in this disclosure is configured to perform the steps of the above-described method for information sharing based on AR glasses.

[0077] In some embodiments, the location acquisition module may include an inertial measurement unit (IMU), a global positioning system (GPS) receiver, and a visual simultaneous localization and mapping processor (V-SLAM).

[0078] By combining the user's absolute position coordinates in a coordinate system with the Earth as the origin provided by GPS, the orientation coordinates provided by the IMU, and the relative displacement acquired by V-SLAM, GPS can provide absolute position to correct and limit V-SLAM drift, while V-SLAM can provide smooth and continuous displacement to compensate for the shortcomings of GPS's low update rate and unstable signal, thus obtaining more accurate and reliable azimuth coordinates.

[0079] In some embodiments, the location acquisition module 2 may include an inertial measurement unit (IMU) and a visual simultaneous localization and mapping processor (V-SLAM). For example, in indoor application scenarios, multiple users share the same local coordinate system established by V-SLAM, and the AR glasses 10 worn by multiple users perform spatial coordinate transformation by aligning with the local coordinate system (e.g., by scanning the same QR code or recognizing the same reference object).

[0080] The following explanation uses a shared concealed target 200 as an example. A first user discovers a disguised enemy personnel during reconnaissance. The first user triggers the main control module 5 of the first AR glasses to lock onto the enemy personnel, i.e., lock onto target 200, via gesture or a button on the first AR glasses. Specifically, the main control module 5 activates the image acquisition module 1 and the location acquisition module 2. The image acquisition module 1 acquires environmental images including target 200 and uses a built-in target 200 detection algorithm (such as YOLO or SSD) to outline target 200 in the image and extract target 200 feature information. This target 200 feature information includes at least one of the following: appearance outline or heat map distribution.

[0081] The location acquisition module 2 acquires the first user's first orientation coordinates, denoted as Pos_A (Xa, Ya, Za, Yawa, Pitcha, Rolla). The main control module 5 calculates the first relative spatial coordinates of the target 200 relative to the first user based on the first orientation coordinates and the image depth information of the environmental image of the target 200 acquired by the image acquisition module 1 (e.g., the image acquisition module 1 includes an RGB camera component and a TOF camera component). For example, the first relative spatial coordinates are a three-dimensional vector in a coordinate system with the first user as the origin, denoted as Coord_T_A (ΔXa, ΔYa, ΔZa). The main control module 5 controls the communication module 4 to send a data packet containing target 200 information (such as the first relative spatial coordinates and target 200 feature information) and the first orientation coordinates to the second AR glasses used by the second user.

[0082] The second user's second AR glasses receive data packets sent by the first AR glasses. The position acquisition module 2 of the second AR glasses obtains the second user's second orientation coordinates, denoted as Pos_B (Xb, Yb, Zb, Yawb, Pitchb, Rollb). The main control module 5 of the second AR glasses performs coordinate transformation calculations based on the second orientation coordinates and the first orientation coordinates, that is, transforming the target 200 from a coordinate system with the first user as the origin to a coordinate system with the second user as the origin. For example, the following simplified transformation process can be used for calculation:

[0083] Coord_T_B ≈ R * (Coord_T_A) + (Pos_A - Pos_B)

[0084] Wherein, Coord_T_B is the second relative spatial coordinate of target 200 obtained by the second AR glasses through the above transformation, denoted as Coord_T_B (ΔXb, ΔYb, ΔZb).

[0085] R is a rotation matrix that takes into account the relative orientation between the first user and the second user. This rotation matrix can be obtained by taking the first orientation rotation matrix of the first user in the global coordinate system and the second orientation rotation matrix of the second user in the global coordinate system, and then performing matrix multiplication on the inverse (or transpose) of the first orientation rotation matrix and the second orientation rotation matrix.

[0086] The main control module 5 of the second AR glasses determines whether the target 200 falls within the display field of view of the augmented display module 3 of the second AR glasses based on the second relative spatial coordinates and the current head orientation of the second user, i.e., orientation coordinates (Yawb, Pitchb, Rollb).

[0087] If it is determined that target 200 falls within the display field of view of the augmented display module 3 of the second AR glasses, the main control module 5 of the second AR glasses first converts the second relative spatial coordinates Coord_T_B into a second virtual display position on the virtual display panel 102 of the second AR glasses, denoted as (Px, Py), based on principles such as perspective projection. Then, based on the actual distance between target 200 and the second user, i.e., the absolute value of the second relative spatial coordinates, i.e., |Coord_T_B|, the module adaptively scales the appearance outline of target 200 in the received target 200 feature information. For example, the greater the actual distance between target 200 and the second user, the smaller the appearance outline of target 200 displayed on the virtual display panel 102, and vice versa. Then, at this (Px, Py) position, the identification feature is presented as an outline icon of a color (e.g., yellow).

[0088] Furthermore, the main control module 5 of the second AR glasses can also control the image acquisition module 1 to acquire the current environmental image of the second user. By cropping an image of a small area centered on the second virtual display position (Px, Py) and extracting its local image features (e.g., using HOG, SIFT or deep learning feature extractors), the extracted local image features are compared with the identification features of the target 200 sent by the first AR glasses to obtain a similarity calculation value.

[0089] If the calculated similarity value is higher than a preset threshold (e.g., 85%), it is determined that the object actually seen by the second user has a high degree of matching with the target 200, that is, the second user can clearly see the target 200. At this time, the second AR glasses present the target 200 on the virtual display panel 102 in a first display mode (e.g., high-saturation red, solid outline).

[0090] If the calculated similarity value is lower than a preset threshold, it can be determined that the object actually seen by the second user does not match the target 200, and there may be an obstruction (such as a wall or bush) between the second user and the target 200. At this time, the second AR glasses present the target 200 on the virtual display panel 102 in a second display mode (e.g., semi-transparent blue, dashed outline, low-frequency flicker).

[0091] If it is determined that target 200 is not within the display field of view of the augmented display module 3 of the second AR glasses, the main control module 5 of the second AR glasses calculates the deviation angle between the direction vector of target 200 and the center line 101 of the second user's field of view, denoted as α, and the distance between target 200 and the second user along the center line 101 of the field of view, denoted as L, and displays a prompt message on the virtual display panel 102 of the second AR glasses, for example, such as... Figure 4 As shown.

[0092] In some embodiments, the target 200 is mobile. In this case, the first user's first AR glasses can be configured to continuously collect environmental images including the target 200 and obtain the target 200's feature information. The main control module 5 is configured to continuously track the target 200 and periodically update the target 200's first relative spatial coordinates. Then, it sends the updated data packet including the first relative spatial coordinates to the second user in real time via the communication module 4. Upon receiving the updated first relative spatial coordinates, the second user's second AR glasses will re-execute the aforementioned steps of calculating the target 200's second relative spatial coordinates and second virtual display position, thereby updating the position of the target 200 displayed on the second user's virtual display panel 102 and reflecting the target 200's movement trajectory in real time.

[0093] It is understood that the main control module 5 of the AR glasses 10 may include at least one processor (such as a CPU or MCU) and a memory. The memory stores computer program instructions. When these program instructions are executed by the processor, they implement the steps of the aforementioned information sharing method based on AR glasses.

[0094] This disclosure also provides a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the above-described method for information sharing based on AR glasses.

[0095] This disclosure also provides a computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the aforementioned information sharing method based on AR glasses when executed by the programmable device.

[0096] In this embodiment, the AR glasses 10, the computer-readable storage medium, and the computer program product are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0097] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0098] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0099] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for information sharing based on AR glasses, characterized in that, include: The first user's first AR glasses acquire the target feature information of the target, the first user's first azimuth coordinates, and the first relative spatial coordinates of the target relative to the first user; The target information and the first location coordinates of the first user are sent to the second AR glasses of at least one second user, wherein the target information includes at least the target's feature information and the first relative spatial coordinates; The second AR glasses calculate the second relative spatial coordinates of the target relative to the second user; Determine whether the second relative spatial coordinates are within the display field of view of the second AR glasses; If the second relative spatial coordinates are within the display field of view of the second AR glasses, then the target is marked on the virtual display panel of the second AR glasses; otherwise, a prompt message for the target is marked on the virtual display panel of the second AR glasses.

2. The method according to claim 1, characterized in that, The method for marking the target on the virtual display panel of the second AR glasses includes: Calculate the display position of the target on the virtual display panel based on the second relative spatial coordinates; The identification features of the target are presented at the display location, wherein the presentation form of the identification features is adaptively scaled according to the distance information in the second relative spatial coordinates.

3. The method according to claim 1 or 2, characterized in that, The method for marking the target on the virtual display panel of the second AR glasses further includes: The second AR glasses capture current environmental images; Extract local image features corresponding to the second relative spatial coordinates from the environmental image; The local image features are compared with the identified features in the received target feature information for similarity. If the similarity is higher than a preset threshold, the target is labeled in the first display mode; If the similarity is lower than the preset threshold, the annotation prompt information will be displayed in the second display mode.

4. The method according to claim 3, characterized in that, The first display mode and the second display mode are distinguished by one or more combinations of color, outline style, transparency, or blink frequency.

5. The method according to claim 1, characterized in that, The prompt information includes the deviation angle and / or straight-line distance of the target relative to the current center of the second user's field of vision.

6. The method according to claim 1, characterized in that, The method further includes: The first AR glasses continuously update data information, which includes at least the first orientation coordinates; The updated data information is sent to the second AR glasses.

7. An AR glasses, characterized in that, include: The image acquisition module is used to acquire images of the user's surrounding environment. The location acquisition module is used to obtain the user's location coordinates; Enhanced display module, used to provide a virtual display panel; The communication module is used for data communication. The main control module is electrically connected to the image acquisition module, the position acquisition module, the enhanced display module and the communication module, and the main control module is configured to perform the steps of the method according to any one of claims 1-6.

8. The AR glasses according to claim 7, characterized in that, The location acquisition module includes an inertial measurement unit, a global positioning system receiver, and a visual simultaneous positioning and mapping processor.

Images