A target sharing method and device for AR glasses based on infrared thermal imaging

Abstract

The present application provides a target sharing method and device for AR glasses based on infrared thermal imaging, relating to the technical field of augmented reality. The method is applied to a second pair of AR glasses and includes: obtaining a first image and a first feature sent by the first pair of AR glasses; obtaining a second image, where the second image is an image captured by a camera of the second pair of AR glasses; obtaining second tag information input by a second user, where the second tag information is used to indicate a second obstacle present in the second image, where the second obstacle includes a second target, and the second user is wearing the second pair of AR glasses; if it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, extracting a second feature of the second target in the second heat map; and if it is determined that the first target and the second target are the same target based on the first and second features, displaying the second target in the second heat map to the second user. The present application can improve the efficiency of hidden target sharing.

Description

Technical Field

[0001] The present application relates to the technical field of augmented reality, and in particular to a target sharing method and device for AR glasses based on infrared thermal imaging. Background Art

[0002] AR glasses, short for augmented reality glasses, are high-tech wearable devices designed to enhance the real-world experience by overlaying virtual information on the user's field of view. They typically consist of a lightweight frame embedded with advanced optical components and display technology. AR glasses are capable of presenting images, videos, data, or other digital visual elements in real time, blending them with the user's physical world. AR glasses utilize cameras, sensors, high-speed processors, and advanced tracking and mapping technologies to analyze the user's environment and adjust the displayed content accordingly, creating an interactive and immersive user experience.

[0003] In the military, AR glasses are seen as a revolutionary technology that can significantly enhance soldiers' battlefield awareness and combat effectiveness. By overlaying critical information such as terrain maps, friend-or-foe identification, target markings, and navigation paths in real time onto a soldier's field of view, AR glasses enable soldiers to make quick tactical decisions while maintaining a comprehensive understanding of their surroundings.

[0004] When performing a mission, soldiers sometimes need to deal with targets hidden behind obstacles, such as enemies hiding in brush. Sometimes, they need to provide information about hidden targets to their teammates so they can provide assistance. However, because the targets are hidden within obstacles, this information must be provided via voice. However, sharing hidden target information through voice is not intuitive, resulting in low efficiency. Therefore, a method to improve the efficiency of hidden target sharing is needed. Summary of the Invention

[0005] The present application provides a target sharing method and device for AR glasses based on infrared thermal imaging, which can improve the efficiency of hidden target sharing.

[0006] In a first aspect of the present application, a target sharing method for AR glasses based on infrared thermal imaging is provided. The method is applied to a second pair of AR glasses, where the second pair of AR glasses is communicatively connected to the first pair of AR glasses. The method includes:

[0007] Obtaining a first image and a first feature sent by the first AR glasses, where the first image is an image captured by a camera of the first AR glasses, a first obstacle containing a first object is present in the first image, and the first feature is a feature of the first object in a first heat map captured by a first infrared thermal imager connected to the first AR glasses;

[0008] Acquire a second image, where the second image is an image captured by a camera of the second AR glasses;

[0009] Obtaining second marking information input by a second user, where the second marking information is used to indicate a second obstacle present in the second image, where the second obstacle includes a second target, and the second user wears the second AR glasses;

[0010] If it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, extracting a second feature of the second object in a second heat map, where the second heat map is captured by a second infrared thermal imager connected to the second AR glasses;

[0011] If, based on the first feature and the second feature, it is determined that the first target and the second target are the same target, the second target in the second heat map is displayed to the second user.

[0012] By adopting the above technical solution, when the first user needs to share the first target contained in the first obstacle with the second user, the first AR glasses will simultaneously send the first image of the first obstacle and the first feature of the first target obtained based on infrared thermal imaging to the second AR glasses. The second AR glasses then use the connected camera and infrared thermal imager, combined with the marking information of the second user, to confirm and match the first target discovered by the first AR glasses. Specifically, it is first determined whether the first obstacle in the first image and the second obstacle in the second image are the same obstacle, and further determined whether the targets in the heat maps captured by the two infrared thermal imagers are the same target. If both judgments are passed, this indicates that the first user and the second user have observed the same target hidden in the obstacle. The second AR glasses realize the sharing of hidden targets by showing the second target in the second heat map to the second user. In short, users can quickly and accurately confirm whether they are observing the same hidden target. This intuitive image sharing and target matching method improves the efficiency of hidden target sharing.

[0013] Optionally, before determining that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, the method further includes:

[0014] extracting a first appearance feature of the first obstacle and a second appearance feature of the second obstacle;

[0015] Obtaining pre-stored appearance features of pre-stored obstacles;

[0016] calculating a first similarity between the first appearance feature and the pre-stored appearance feature, and a second similarity between the second appearance feature and the pre-stored appearance feature;

[0017] Determining whether the first similarity is less than a preset similarity threshold, and determining whether the second similarity is less than a preset similarity threshold;

[0018] If it is determined that the first similarity is greater than the preset similarity threshold, and the second similarity is greater than the preset similarity threshold, then it is determined that the first obstacle and the second obstacle are the same obstacle;

[0019] If it is determined that the first similarity is less than or equal to the preset similarity threshold, or the second similarity is less than or equal to the preset similarity threshold, it is determined that the first obstacle and the second obstacle are not the same obstacle.

[0020] By employing this technical solution, the two users can automatically determine whether they are observing the same obstacle by calculating and comparing the similarity between the obstacle's appearance features and those in a pre-stored database, given their different perspectives. This automated process significantly reduces the need for manual judgment, thereby improving processing speed and real-time performance.

[0021] Optionally, extracting the second feature of the second target in the second heat map specifically includes:

[0022] Preprocessing the second thermal map to obtain a processed image;

[0023] performing temperature distribution analysis on the processed image to obtain a plurality of regions;

[0024] determining a first region and a second region among the plurality of regions, the first region and the second region having different temperatures;

[0025] If it is determined that the temperature of the first area is higher than the temperature of the second area, determining the first area as the area where the second target is located;

[0026] A first temperature distribution characteristic of the first region is determined as the second characteristic.

[0027] By adopting the above technical solution, traditional visual recognition methods may not be effective in certain environmental conditions, such as low light or obstructed vision. This temperature distribution-based recognition method can better adapt to these conditions. By analyzing the temperature distribution of different areas through heat maps, it can more accurately identify the target area, because the human body usually has different temperature characteristics from the surrounding environment. In addition, by extracting the temperature distribution characteristics of the target area, it is easier to subsequently analyze whether the targets in two heat maps are the same target.

[0028] Optionally, if it is determined based on the first feature and the second feature that the first target and the second target are the same target, before presenting the second target in the second heat map to the second user, the method further includes:

[0029] Calculating a first average temperature based on the first temperature distribution characteristic;

[0030] calculating a second average temperature based on a second temperature distribution characteristic, where the second temperature distribution characteristic is a temperature distribution characteristic of the first target in the first thermal map;

[0031] It is determined whether the temperature difference between the first average temperature and the second average temperature is less than a preset temperature difference, thereby determining whether the first feature is the same as the second feature.

[0032] By adopting the above technical solution, by calculating the average temperature of two targets and comparing their difference, if the difference is less than a preset threshold, it can be more reliably judged that the two thermal map targets are actually the same. This method can reduce misjudgments caused by different observation angles and is also suitable for judging hidden targets because hidden targets cannot be judged by visual methods.

[0033] Optionally, presenting the second target in the second heat map to the second user specifically includes:

[0034] determining a target area corresponding to the second target in the second heat map;

[0035] The target area is marked with a preset frame and displayed.

[0036] By adopting the above technical solution, by marking the target area with a frame, the position of the second target can be intuitively highlighted, making it easier for the second user to identify and pay attention to the shared target.

[0037] Optionally, the first image sent by the first AR glasses is obtained, wherein after the first AR glasses obtain the first image, first marking information input by the first user is obtained, and the first marking information is used to indicate a first obstacle present in the first image, and the first obstacle includes the first target.

[0038] Optionally, the first feature sent by the first AR glasses is obtained, wherein the first AR glasses obtain the first heat map, perform feature extraction on the first target in the first heat map, and obtain the first feature.

[0039] In a second aspect of the present application, a target sharing device for AR glasses based on infrared thermal imaging is provided. The device is a second pair of AR glasses, including an acquisition module, a judgment module, and a processing module, wherein:

[0040] The acquisition module is configured to acquire a first image and a first feature sent by the first pair of AR glasses, where the first image is an image captured by a camera of the first pair of AR glasses, and the first feature is a feature of a first target present in a first heat map captured by a first infrared thermal imager connected to the first pair of AR glasses;

[0041] The acquisition module is configured to acquire a second image, where the second image is an image captured by a camera of the second AR glasses;

[0042] The acquisition module is configured to acquire second marking information input by a second user, where the second marking information is used to indicate a second obstacle present in the second image, where the second obstacle includes a second target, and the second user wears the second AR glasses;

[0043] The judgment module is configured to, if it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, extract a second feature of the second object in a second heat map, where the second heat map is captured by a second infrared thermal imager connected to the second AR glasses;

[0044] The processing module is configured to, based on the first feature and the second feature, display the second target in the second heat map to the second user if it is determined that the first target and the second target are the same target.

[0045] Optionally, the processing module is used to extract a first appearance feature of the first obstacle and a second appearance feature of the second obstacle;

[0046] The acquisition module is used to acquire pre-stored appearance features of pre-stored obstacles;

[0047] The processing module is configured to calculate a first similarity between the first appearance feature and the pre-stored appearance feature, and a second similarity between the second appearance feature and the pre-stored appearance feature;

[0048] The judgment module is used to judge whether the first similarity is less than a preset similarity threshold, and judge whether the second similarity is less than a preset similarity threshold;

[0049] The judgment module is configured to determine that the first obstacle and the second obstacle are the same obstacle if it is determined that the first similarity is greater than the preset similarity threshold and the second similarity is greater than the preset similarity threshold;

[0050] The judgment module is configured to determine that the first obstacle and the second obstacle are not the same obstacle if it is determined that the first similarity is less than or equal to the preset similarity threshold, or the second similarity is less than or equal to the preset similarity threshold.

[0051] Optionally, the processing module is used to preprocess the second thermal map to obtain a processed image;

[0052] The processing module is used to perform temperature distribution analysis on the processed image to obtain multiple regions;

[0053] The judgment module is used to determine a first area and a second area in the plurality of areas, wherein the first area has a different temperature from the second area;

[0054] The judgment module is configured to, if it is determined that the temperature of the first area is higher than the temperature of the second area, determine that the first area is the area where the second target is located;

[0055] The judgment module is configured to determine that the first temperature distribution feature of the first area is the second feature.

[0056] Optionally, the processing module is configured to calculate a first average temperature based on the first temperature distribution characteristic;

[0057] The processing module is configured to calculate a second average temperature based on a second temperature distribution characteristic, where the second temperature distribution characteristic is a temperature distribution characteristic of the first target in the first thermal map;

[0058] The judgment module is used to judge whether the temperature difference between the first average temperature and the second average temperature is less than a preset temperature difference, thereby determining whether the first feature is the same as the second feature.

[0059] Optionally, the judgment module is used to determine a target area corresponding to the second target in the second heat map;

[0060] The processing module is used to mark the target area with a preset picture frame and display it.

[0061] Optionally, the acquisition module is used to acquire the first image sent by the first AR glasses, wherein, after the first AR glasses acquire the first image, they acquire first marking information input by the first user, and the first marking information is used to indicate a first obstacle present in the first image, and the first obstacle includes the first target.

[0062] Optionally, the acquisition module is used to acquire the first feature sent by the first AR glasses, wherein the first AR glasses acquire the first heat map, perform feature extraction on the first target in the first heat map, and obtain the first feature.

[0063] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0064] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0065] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0066] When the first user needs to share the first target contained in the first obstacle with the second user, the first AR glasses will simultaneously send the first image of the first obstacle and the first feature of the first target obtained based on infrared thermal imaging to the second AR glasses. The second AR glasses then use the connected camera and infrared thermal imager, combined with the tag information of the second user, to confirm and match the first target discovered by the first AR glasses. Specifically, it is first determined whether the first obstacle in the first image and the second obstacle in the second image are the same obstacle, and further determined whether the targets in the heat maps captured by the two infrared thermal imagers are the same target. If both judgments are passed, this indicates that the first user and the second user have observed the same target hidden in the obstacle. The second AR glasses realize the sharing of hidden targets by showing the second target in the second heat map to the second user. In short, users can quickly and accurately confirm whether they are observing the same hidden target. This intuitive image sharing and target matching method improves the efficiency of hidden target sharing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of a target sharing method for AR glasses based on infrared thermal imaging disclosed in an embodiment of the present application;

[0068] Figure 2 This is a module diagram of a target sharing device for AR glasses based on infrared thermal imaging disclosed in an embodiment of the present application;

[0069] Figure 3 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0070] Explanation of the reference numerals: 201, acquisition module; 202, judgment module; 203, processing module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0072] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0073] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0074] AR glasses, short for augmented reality glasses, are high-tech wearable devices designed to enhance the real-world experience by overlaying virtual information on the user's field of view. They typically consist of a lightweight frame embedded with advanced optical components and display technology. AR glasses are capable of presenting images, videos, data, or other digital visual elements in real time, blending them with the user's physical world. AR glasses utilize cameras, sensors, high-speed processors, and advanced tracking and mapping technologies to analyze the user's environment and adjust the displayed content accordingly, creating an interactive and immersive user experience.

[0075] In the military, AR glasses are seen as a revolutionary technology that can significantly enhance soldiers' battlefield awareness and combat effectiveness. By overlaying critical information such as terrain maps, friend-or-foe identification, target markings, and navigation paths in real time onto a soldier's field of view, AR glasses enable soldiers to make quick tactical decisions while maintaining a comprehensive understanding of their surroundings.

[0076] When performing a mission, soldiers sometimes need to deal with targets hidden behind obstacles, such as enemies hiding in brush. Sometimes, they need to provide information about hidden targets to their teammates so they can provide assistance. However, because the targets are hidden within obstacles, this information must be provided via voice. However, sharing hidden target information through voice is not intuitive, resulting in low efficiency. Therefore, a method to improve the efficiency of hidden target sharing is needed.

[0077] This embodiment discloses a target sharing method for AR glasses based on infrared thermal imaging. Figure 1 , including the following steps S110-S150:

[0078] S110: Acquire a first image and a first feature sent by the first AR glasses.

[0079] An embodiment of the present application discloses a target sharing method for AR glasses based on infrared thermal imaging, which is applied to a second pair of AR glasses. The second pair of AR glasses is communicatively connected to the first pair of AR glasses, wherein a first user wears the first pair of AR glasses and a second user wears the second pair of AR glasses.

[0080] When a first user observes their surroundings through the first pair of AR glasses, the built-in camera in the first pair of AR glasses captures and displays the image of their current field of view in real time. When the first user discovers a hidden target (first target) hidden within an obstacle, they align their line of sight with the obstacle, causing the camera in the first pair of AR glasses to capture the image and obtain the first image. The first target is preferably a person hidden within or behind an obstacle, which can be grass, rocks, or a house. The first user also needs to mark the first image to assist the first pair of AR glasses in identifying the first obstacle within the first image that contains the first target. First, the first user inputs first marking information into the first pair of AR glasses. This can be done through the first pair of AR glasses' interactive interface. Interaction can be achieved in various ways, such as voice commands, gesture control, or using a connected handheld device. For details on how to obtain the first marking information, please refer to the following section on how the second pair of AR glasses obtains the second marking information of the second user.

[0081] Since the first target is hidden within the first obstacle, in order for the second user to accurately identify the hidden first target, a first infrared thermal imager is required to capture a thermal image. This allows the first infrared thermal imager to penetrate the obstacle and capture the target image within it. Infrared thermal imaging is a technology that uses infrared radiation to capture and create thermal images of objects. Infrared thermal imagers use specialized optical components and optoelectronic processing technology to capture infrared radiation and convert it into a visible thermal image. The operating principle of an infrared thermal imager is that infrared light passes through a specific optical lens, is absorbed by an infrared detector, and is converted into an electrical signal, which in turn forms a thermal image observable to the human eye.

[0082] The first AR glasses obtain a first thermal image captured by a first infrared thermal imager, preprocess the first thermal image to obtain a processed image; perform temperature distribution analysis on the processed image to obtain multiple regions; determine a first region and a second region among the multiple regions, where the temperature of the first region is different from that of the second region; if it is determined that the temperature of the first region is higher than the temperature of the second region, determine that the first region is the region where the first target is located; and determine that the temperature distribution feature of the first region is the first feature of the first target.

[0083] Specifically, the first AR glasses perform denoising, which uses a digital filter to remove random noise from the heat map. Image enhancement adjusts contrast and brightness to make temperature differences more apparent. Sharpening improves the clarity of the heat map, making edges and features more prominent. Through these preprocessing steps, the first AR glasses produce a processed image.

[0084] Because the human body emits thermal radiation, it typically exhibits a temperature signature different from that of its surroundings in infrared thermal imaging. By analyzing the temperature distribution of the thermal map and identifying warmer areas in the image, the approximate location of the human body can be determined. Image processing techniques are used to analyze the temperature distribution of the thermal map and convert it into a temperature matrix, where each pixel represents a temperature value. Image segmentation techniques, such as threshold-based segmentation methods or clustering algorithms, are then used to identify areas with significantly different temperatures, areas with temperatures that are within the same range, or areas that are identical.

[0085] For the first area and the second area in the multiple areas, the temperature of the first area is compared with the temperature of the second area. If the first area is the area corresponding to the first target in reality, that is, the person hidden in the obstacle, then the overall temperature of the first area should be higher than the overall temperature of the second area. The first area determined to correspond to the first target is further analyzed, and its first temperature distribution feature is extracted by identifying specific thermal map patterns and calculating the average temperature, temperature gradient, temperature peak and other features within the area to obtain the first feature. Finally, the first AR glasses send the first image and the first feature to the second AR glasses, so that the second AR glasses obtain the first image and the first feature.

[0086] S120: Acquire a second image.

[0087] After the second AR glasses obtain the first image and first feature of the first AR glasses, it indicates that the first user has a goal to share with the second user. When the second user observes the surrounding environment through the second AR glasses, the built-in camera of the second AR glasses will capture and display the image of the current field of view in real time.

[0088] S130: Acquire second marking information input by a second user.

[0089] When the second AR glasses capture an image (second image) and there is an obstacle in the second image, if the second user needs to confirm it, it is determined whether the obstacle in the second image contains the target that the first user needs to share. First, the second marking information is input into the second AR glasses. If the second AR glasses are equipped with a touchpad or touch-sensitive area, the second user can operate through touch gestures, such as tapping, sliding, or long pressing to select and mark a specific area in the image, thereby marking the obstacle in the second image. Eye tracking technology can detect the user's line of sight and eye movement. The second user can mark the specific area (obstacle area) by looking at it and performing a specific action (such as blinking), so that the second AR glasses can obtain the second marking information.

[0090] Then, the second AR glasses determine whether the first obstacle in the first image and the second obstacle in the second image are the same obstacle. Since the first user and the second user observe the first obstacle from different angles, the first obstacle in the first image cannot be exactly the same as the second obstacle in the second image. Whether the first obstacle and the second obstacle are the same obstacle can only be determined by the similarity of appearance features, thereby determining whether the first obstacle and the second obstacle are actually the same obstacle.

[0091] First, the second AR glasses use image processing techniques, such as edge detection, color histogram analysis, and texture analysis, to analyze the images of the first and second obstacles and extract their appearance features. These features may include shape, size, color, texture, and edge information. Appearance feature extraction for the first obstacle yields a first appearance feature for the first obstacle, and appearance feature extraction for the second obstacle yields a second appearance feature for the second obstacle. Appearance features of known obstacles are then retrieved from a database or pre-stored data. These pre-stored features are previously analyzed features of known obstacle types. Appropriate algorithms, such as Euclidean distance, cosine similarity, and Hamming distance, are then used to calculate a first similarity between the first obstacle and the pre-stored features, and a second similarity between the second obstacle and the pre-stored features. Specifically, the first similarity between the first appearance feature and the pre-stored features is calculated, and the second similarity between the second appearance feature and the pre-stored features is calculated. A preset similarity threshold is set to determine whether two obstacles are sufficiently similar to be considered the same. This threshold can be adjusted based on experience, historical data, or experimentation.

[0092] If the first similarity is greater than the preset similarity threshold, it indicates that the first obstacle and the pre-stored obstacle are similar in appearance and are the same type of obstacle. If the second similarity is greater than the preset similarity threshold, it indicates that the second obstacle and the pre-stored obstacle are similar in appearance and are the same type of obstacle. Therefore, the first and second obstacles are the same obstacle. Conversely, if the first similarity is less than or equal to the preset similarity threshold, or if the second similarity is less than or equal to the preset similarity threshold, it cannot be determined that the first and second obstacles are the same obstacle.

[0093] Because the two users observe from different perspectives, the system automatically determines whether the two users are observing the same obstacle by calculating and comparing the similarity between the obstacle's appearance features and those in a pre-stored database. This automated process significantly reduces the need for manual judgment, thereby improving processing speed and real-time performance.

[0094] S130: If it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, extract a second feature of the second target in the second heat map.

[0095] If the second AR glasses determine that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, it indicates that the first user and the second user may have observed the same obstacle, and infrared thermal imaging is further required to determine the hidden target in the obstacle.

[0096] First, the second AR glasses extract features from the second thermal image captured by the second infrared thermal imager. The second AR glasses then perform preprocessing steps such as denoising, image enhancement, and sharpening on the second thermal image to produce a processed image. Image processing techniques are used to analyze the temperature distribution of the thermal image, and image segmentation techniques are used to identify areas in the second thermal image with significantly different temperatures. The temperatures in each area are within the same range or are completely identical.

[0097] For a first region and a second region in the multiple regions, the temperature of the first region is compared with the temperature of the second region. If the first region corresponds to the actual second target, that is, a person hidden in an obstacle, then the overall temperature of the first region should be higher than the overall temperature of the second region. The first region determined to correspond to the second target is further analyzed to extract its second temperature distribution feature to obtain the second feature.

[0098] Traditional visual recognition methods may not be effective in certain environmental conditions, such as low light or obstructed vision. This temperature distribution-based recognition method can better adapt to these conditions. By analyzing the temperature distribution of different regions in a heat map, it can more accurately identify the target area, as the human body typically has different temperature characteristics from the surrounding environment. Furthermore, by extracting the temperature distribution characteristics of the target area, it is easier to subsequently determine whether the objects in two heat maps are the same.

[0099] By calculating the average temperature of two targets and comparing their difference, if the difference is less than a preset threshold, it can be more reliably determined that the two heat map targets are actually the same. This method can reduce misjudgments caused by different observation angles and is also suitable for judging hidden targets because hidden targets cannot be judged by visual methods.

[0100] S150: If it is determined that the first target and the second target are the same target based on the first feature and the second feature, the second target in the second heat map is displayed to the second user.

[0101] Then, the similarity between the first feature and the second feature is determined. First, based on the first temperature distribution feature, that is, the temperature values ​​at different positions in the selected area in the first heat map, the average value is calculated to obtain the first average temperature. Similarly, based on the second temperature distribution feature, that is, the temperature values ​​at different positions in the selected area in the second heat map, the average value is calculated to obtain the second average temperature. The difference between the first average temperature and the second average temperature is calculated to obtain the temperature difference. Prior to this, a preset temperature difference threshold is set to determine whether the first temperature distribution feature and the second temperature distribution feature are similar. If the calculated temperature difference is less than the preset temperature difference, it can be determined that the first temperature distribution feature is the same as the second temperature distribution feature, that is, the first feature is the same as the second feature. If the calculated temperature difference is greater than or equal to the preset temperature difference, it is determined that the first temperature distribution feature is different from the second temperature distribution feature, and the first feature is different from the second feature.

[0102] When the calculated temperature difference is less than the preset temperature difference, it is determined that the first temperature distribution feature is the same as the second temperature distribution feature, that is, the first feature is the same as the second feature. This further indicates that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, and the first target in the first obstacle and the second target in the second obstacle are the same target, and the second target is the first target that the first user needs to share. The second AR glasses then display the second target to the second user in a preset manner, and because the second target is hidden in the obstacle, it needs to be displayed through a heat map. First, the second AR glasses use a preset frame to mark the target area corresponding to the second target in the second heat map determined in step S130, and display it on the display lens of the AR glasses. By marking the target area with a frame, the position of the second target can be intuitively highlighted, making it easier for the second user to identify and pay attention to the shared target.

[0103] By adopting the above technical solution, when the first user needs to share the first target contained in the first obstacle with the second user, the first AR glasses will simultaneously send the first image of the first obstacle and the first feature of the first target obtained based on infrared thermal imaging to the second AR glasses. The second AR glasses then use the connected camera and infrared thermal imager, combined with the marking information of the second user, to confirm and match the first target discovered by the first AR glasses. Specifically, it is first determined whether the first obstacle in the first image and the second obstacle in the second image are the same obstacle, and further determined whether the targets in the heat maps captured by the two infrared thermal imagers are the same target. If both judgments are passed, this indicates that the first user and the second user have observed the same target hidden in the obstacle. The second AR glasses realize the sharing of hidden targets by showing the second target in the second heat map to the second user. In short, users can quickly and accurately confirm whether they are observing the same hidden target. This intuitive image sharing and target matching method improves the efficiency of hidden target sharing.

[0104] This embodiment further discloses a target sharing device for AR glasses based on infrared thermal imaging. The device is a second pair of AR glasses and includes an acquisition module 201, a judgment module 202, and a processing module 203, wherein:

[0105] The acquisition module 201 is used to obtain the first image and the first feature sent by the first AR glasses. The first image is the image captured by the camera of the first AR glasses. The first feature is the feature of the first target in the first heat map. The first heat map is captured by the first infrared thermal imager connected to the first AR glasses.

[0106] The acquisition module 201 is used to acquire a second image, where the second image is an image captured by a camera of the second AR glasses.

[0107] The acquisition module 201 is used to obtain second marking information input by a second user, where the second marking information is used to indicate a second obstacle present in the second image, where the second obstacle includes a second target, and the second user wears second AR glasses.

[0108] The judgment module 202 is used to extract the second feature of the second target in the second heat map if it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle. The second heat map is collected by a second infrared thermal imager connected to the second AR glasses.

[0109] The processing module 203 is configured to, based on the first feature and the second feature, display the second target in the second heat map to the second user if it is determined that the first target and the second target are the same target.

[0110] In a possible implementation, the processing module 203 is configured to extract a first appearance feature of the first obstacle and a second appearance feature of the second obstacle.

[0111] The acquisition module 201 is used to acquire pre-stored appearance features of pre-stored obstacles.

[0112] The processing module 203 is configured to calculate a first similarity between the first appearance feature and the pre-stored appearance feature, and a second similarity between the second appearance feature and the pre-stored appearance feature.

[0113] The determination module 202 is configured to determine whether the first similarity is less than a preset similarity threshold, and determine whether the second similarity is less than a preset similarity threshold.

[0114] The judgment module 202 is configured to determine that the first obstacle and the second obstacle are the same obstacle if it is determined that the first similarity is greater than a preset similarity threshold and the second similarity is greater than the preset similarity threshold.

[0115] The judgment module 202 is configured to determine that the first obstacle and the second obstacle are not the same obstacle if it is determined that the first similarity is less than or equal to a preset similarity threshold, or the second similarity is less than or equal to the preset similarity threshold.

[0116] In a possible implementation, the processing module 203 is configured to preprocess the second thermal map to obtain a processed image.

[0117] The processing module 203 is used to perform temperature distribution analysis on the processed image to obtain multiple regions.

[0118] The judgment module 202 is configured to determine a first area and a second area among the multiple areas, where the first area has a different temperature from the second area.

[0119] The judgment module 202 is configured to determine that the first area is the area where the second target is located if it is determined that the temperature of the first area is higher than the temperature of the second area.

[0120] The judgment module 202 is configured to determine that the first temperature distribution feature of the first region is the second feature.

[0121] In a possible implementation, the processing module 203 is configured to calculate a first average temperature based on the first temperature distribution characteristic.

[0122] The processing module 203 is configured to calculate a second average temperature based on a second temperature distribution feature, where the second temperature distribution feature is a temperature distribution feature of the first target in the first thermal map.

[0123] The judgment module 202 is configured to judge whether a temperature difference between the first average temperature and the second average temperature is less than a preset temperature difference, thereby determining whether the first feature is the same as the second feature.

[0124] In a possible implementation, the determination module 202 is configured to determine a target area corresponding to the second target in the second heat map.

[0125] The processing module 203 is used to mark the target area with a preset frame and display it.

[0126] In one possible embodiment, the acquisition module 201 is used to acquire a first image sent by a first AR glasses, wherein after the first AR glasses acquire the first image, they acquire first marking information input by a first user, and the first marking information is used to indicate a first obstacle present in the first image, and the first obstacle includes a first target.

[0127] In a possible implementation, the acquisition module 201 is configured to acquire a first feature sent by the first AR glasses, wherein the first AR glasses acquire a first heat map, perform feature extraction on a first target in the first heat map, and obtain the first feature.

[0128] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0129] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0130] The communication bus 302 is used to implement the connection and communication between these components.

[0131] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0132] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0133] The processor 301 may include one or more processing cores. The processor 301 utilizes various interfaces and circuits to connect various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 305, as well as accesses data stored in the memory 305, to perform various server functions and process data. Optionally, the processor 301 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 301.

[0134] The memory 305 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 305 may include non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch control, sound playback, image playback, etc.), and instructions for implementing the aforementioned method embodiments. The data storage area may store data related to the aforementioned method embodiments. The memory 305 may also optionally be at least one storage device located remotely from the processor 301. As shown in the figure, the memory 305, as a computer storage medium, may include an operating system, a network communication module, a user interface 303 module, and an application program for a method for sharing objects in AR glasses based on infrared thermal imaging.

[0135] exist Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application stored in the memory 305 for a target sharing method of AR glasses based on infrared thermal imaging. When executed by one or more processors 301, the electronic device executes one or more methods in the above embodiments.

[0136] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0137] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0139] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory 305. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a memory 305, includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory 305 includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0142] The present application also discloses a computer-readable storage medium storing instructions, which, when executed by one or more processors 301, enable an electronic device to execute one or more of the methods described in the above embodiments.

[0143] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A target sharing method for AR glasses based on infrared thermal imaging, characterized in that: The method is applied to a second pair of AR glasses, where the second pair of AR glasses is communicatively connected to the first pair of AR glasses. The method includes: Obtaining a first image and a first feature sent by the first AR glasses, where the first image is an image captured by a camera of the first AR glasses, a first obstacle containing a first object is present in the first image, and the first feature is a feature of the first object in a first heat map captured by a first infrared thermal imager connected to the first AR glasses; Acquire a second image, where the second image is an image captured by a camera of the second AR glasses; Obtaining second marking information input by a second user, where the second marking information is used to indicate a second obstacle present in the second image, where the second obstacle includes a second target, and the second user wears the second AR glasses; If it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, extracting a second feature of the second object in a second heat map, where the second heat map is captured by a second infrared thermal imager connected to the second AR glasses; If, based on the first feature and the second feature, it is determined that the first target and the second target are the same target, the second target in the second heat map is displayed to the second user.

2. The target sharing method of AR glasses based on infrared thermal imaging according to claim 1, characterized in that: Before determining that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, the method further includes: extracting a first appearance feature of the first obstacle and a second appearance feature of the second obstacle; Obtaining pre-stored appearance features of pre-stored obstacles; calculating a first similarity between the first appearance feature and the pre-stored appearance feature, and a second similarity between the second appearance feature and the pre-stored appearance feature; Determining whether the first similarity is less than a preset similarity threshold, and determining whether the second similarity is less than a preset similarity threshold; If it is determined that the first similarity is greater than the preset similarity threshold, and the second similarity is greater than the preset similarity threshold, then it is determined that the first obstacle and the second obstacle are the same obstacle; If it is determined that the first similarity is less than or equal to the preset similarity threshold, or the second similarity is less than or equal to the preset similarity threshold, it is determined that the first obstacle and the second obstacle are not the same obstacle.

3. The target sharing method of AR glasses based on infrared thermal imaging according to claim 1, characterized in that: Extracting the second feature of the second target in the second heat map specifically includes: Preprocessing the second thermal map to obtain a processed image; performing temperature distribution analysis on the processed image to obtain a plurality of regions; determining a first region and a second region among the plurality of regions, the first region and the second region having different temperatures; If it is determined that the temperature of the first area is higher than the temperature of the second area, determining the first area as the area where the second target is located; A first temperature distribution characteristic of the first region is determined as the second characteristic.

4. The target sharing method of AR glasses based on infrared thermal imaging according to claim 3, characterized in that: If, based on the first feature and the second feature, it is determined that the first target and the second target are the same target, before presenting the second target in the second heat map to the second user, the method further includes: Calculating a first average temperature based on the first temperature distribution characteristic; calculating a second average temperature based on a second temperature distribution characteristic, where the second temperature distribution characteristic is a temperature distribution characteristic of the first target in the first thermal map; It is determined whether the temperature difference between the first average temperature and the second average temperature is less than a preset temperature difference, thereby determining whether the first feature is the same as the second feature.

5. The target sharing method of AR glasses based on infrared thermal imaging according to claim 1, characterized in that: The presenting the second target in the second heat map to the second user specifically includes: determining a target area corresponding to the second target in the second heat map; The target area is marked with a preset frame and displayed.

6. The target sharing method of AR glasses based on infrared thermal imaging according to claim 1, characterized in that: The method further comprises: Obtain the first image sent by the first AR glasses, wherein, after the first AR glasses obtain the first image, obtain first marking information input by a first user, where the first marking information is used to indicate a first obstacle present in the first image, and the first obstacle includes the first target.

7. The target sharing method of AR glasses based on infrared thermal imaging according to claim 1, characterized in that: The method further comprises: Obtain the first feature sent by the first AR glasses, wherein the first AR glasses obtain the first heat map, perform feature extraction on the first target in the first heat map, and obtain the first feature.

8. A target sharing device for AR glasses based on infrared thermal imaging, characterized in that: The device is a second AR glasses, comprising an acquisition module (201), a judgment module (202) and a processing module (203), wherein: The acquisition module (201) is used to acquire a first image and a first feature sent by a first pair of AR glasses, wherein the first image is an image captured by a camera of the first pair of AR glasses, and the first feature is a feature of a first target present in a first heat map, wherein the first heat map is captured by a first infrared thermal imager connected to the first pair of AR glasses; The acquisition module (201) is used to acquire a second image, where the second image is an image captured by a camera of the second AR glasses; The acquisition module (201) is used to acquire second marking information input by a second user, the second marking information is used to indicate a second obstacle present in the second image, the second obstacle includes a second target, and the second user wears the second AR glasses; The judgment module (202) is configured to extract a second feature of the second target in a second heat map if it is determined that the first obstacle in the first image and the second obstacle in the second image are the same obstacle, the second heat map being collected by a second infrared thermal imager connected to the second AR glasses; The processing module (203) is used to, based on the first feature and the second feature, if it is determined that the first target and the second target are the same target, display the second target in the second heat map to the second user.

9. An electronic device, characterized in that: The electronic device comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are both used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images