Image recognition optimization method and device, ar glasses and storage medium

Abstract

The application provides an image recognition optimization method and device, AR glasses and a storage medium, relates to the technical field of image processing, and solves the problem of poor recognition accuracy of AR glasses in a poor background light environment. The AR glasses perform grayscale processing on an image obtained in an image, and then obtain a grayscale ratio value and a weighted average value corresponding to an image feature point to determine a corresponding quality parameter. If the quality parameter is less than a preset parameter threshold, it is determined that the current RGB camera is not adapted to the current light environment, and the shooting parameters of the RGB camera are adjusted to adapt to the current light environment. The scheme can dynamically adapt the image recognition of the AR glasses in different light environments, so that the image output by the AR glasses satisfies the maximum gradient information of the feature points, thereby improving the recognition accuracy of the AR glasses.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image recognition optimization method, apparatus, AR glasses, and storage medium. Background Technology

[0002] With the advancement of digital cultural tourism, some small museums, such as exhibition halls and memorial halls, or museum exhibition halls, are promoting the implementation of digital displays to present cultural relics or exhibits to visitors in a more vivid and engaging way. For example, digital equipment such as AR (Augmented Reality) glasses are placed next to the display cases. These glasses can perform image recognition on the exhibits and present digital models of the exhibits to visitors wearing them, making the exhibits more vivid and technologically advanced.

[0003] AR glasses are equipped with RGB cameras. The images captured by RGB cameras are mainly composed of a mixture of red, green, and blue primary colors of light, and the image quality is closely related to the ambient light. However, museums are typically in low-light environments, while exhibits and artifacts are illuminated with strong light to create a brighter environment around them. AR glasses have good recognition rates under normal lighting conditions, but in low-light or high-light environments, RGB images tend to be too dark or overexposed, resulting in poor recognition accuracy. Summary of the Invention

[0004] This application provides an image recognition optimization method, device, AR glasses, and storage medium, which solves the problem of poor recognition accuracy of AR glasses in poor background light environments in related technologies. This solution can dynamically adapt to the image recognition of AR glasses under different lighting environments, so that the image output by AR glasses satisfies the maximum gradient information of feature points, thereby improving the recognition accuracy of AR glasses.

[0005] In a first aspect, this application provides an image recognition optimization method applied to AR glasses, the AR glasses including an RGB camera for acquiring RGB image stream data, the image recognition optimization method including:

[0006] The target images in the RGB image stream data are converted to grayscale to obtain the corresponding multi-frame grayscale images.

[0007] Confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group. The grayscale image group includes multiple frames of grayscale images after grayscale processing.

[0008] Determine the quality parameters corresponding to the grayscale image group based on the weighted average value and the grayscale ratio;

[0009] If the quality parameters are less than the preset threshold, adjust the shooting parameters of the RGB camera. The preset threshold is the quality parameters obtained in advance based on the images captured by the RGB camera under normal lighting conditions.

[0010] Therefore, AR glasses perform grayscale processing on the acquired image to obtain the grayscale ratio and the weighted average of the corresponding image feature points, thus determining the corresponding quality parameters. If the quality parameters are lower than a preset threshold, it can be determined that the current RGB camera is not adapted to the current lighting environment, and the shooting parameters of the RGB camera can be adjusted to suit the lighting conditions. In this way, AR glasses can dynamically adapt to different lighting environments by scoring the acquired image quality and adjusting the RGB camera when the image quality does not meet the requirements. This allows the AR glasses to output images that maximize the gradient information of feature points, thereby improving the accuracy of image recognition under different lighting conditions and providing users with better image recognition results.

[0011] In one embodiment, the weighted average value corresponding to image feature points of the grayscale image group and the grayscale ratio value of the grayscale image group are determined. The grayscale image group includes multiple frames of grayscale images after grayscale processing, including:

[0012] Feature points are extracted from each frame of grayscale image to obtain the image feature points and grayscale values ​​of each image feature point, so as to determine the weighted average value of the grayscale image group.

[0013] Perform grayscale statistics on each frame of grayscale image to obtain the grayscale ratio value of the grayscale image group.

[0014] Therefore, by extracting feature points from each grayscale image within a grayscale image group and performing grayscale statistics, AR glasses can determine the image quality of the grayscale image group, and then adjust the RGB camera to adapt to the current lighting environment, thereby helping to improve the image recognition accuracy of AR glasses and making the image recognition effect of AR glasses better.

[0015] In one embodiment, feature point extraction is performed on each frame of grayscale image to obtain the image feature points and grayscale values ​​of the image feature points of each frame of grayscale image, in order to determine the weighted average value of the grayscale image group, including:

[0016] Based on the preset feature point detection algorithm, feature points are extracted from all grayscale images to determine the image feature points on the grayscale images, and the grayscale value corresponding to the image feature point is used as the feature point value of the image feature point.

[0017] The average value of the feature points of all image feature points in each frame of grayscale image is calculated to determine the average value of the feature points of each frame of grayscale image.

[0018] Based on the weighting coefficients corresponding to each grayscale image, a weighted average is calculated on the average value of the feature points of all grayscale images to determine the weighted average value of the grayscale image group.

[0019] Therefore, AR glasses calculate the weighted average of grayscale image groups to obtain the quality score of the images acquired by the RGB camera. This allows the AR glasses to adjust the RGB camera in a timely manner, enabling the RGB camera to dynamically adapt to the image recognition of AR glasses under different lighting conditions, which is beneficial to improving the recognition accuracy of AR glasses.

[0020] In one embodiment, grayscale statistics are performed on each frame of grayscale image to obtain the grayscale ratio value of the grayscale image group, including:

[0021] Based on the grayscale statistical results, determine the grayscale value of all pixels in each frame of grayscale image;

[0022] The average grayscale value of each grayscale image is determined based on the grayscale value of all pixels in each grayscale image and the number of pixels.

[0023] Based on the weighting coefficients corresponding to each grayscale image, a weighted average is calculated on the grayscale average values ​​of all grayscale images to determine the grayscale ratio of the grayscale image group.

[0024] Therefore, AR glasses calculate the grayscale ratio of grayscale image groups to obtain the quality score of the image acquired by the RGB camera. This allows the AR glasses to adjust the RGB camera in a timely manner, enabling the RGB camera to dynamically adapt to the image recognition of AR glasses under different lighting conditions, which is beneficial to improving the recognition accuracy of AR glasses.

[0025] In one embodiment, the grayscale statistics result is a grayscale histogram. Therefore, AR glasses can obtain the grayscale values ​​corresponding to all pixels in each frame of grayscale image, and can also determine the number of pixels at each grayscale value, thereby helping to improve the efficiency of grayscale statistics.

[0026] In one embodiment, the first weight corresponds to the weighted average value, the second weight corresponds to the grayscale ratio value, and the sum of the first weight and the second weight is 1;

[0027] Based on the weighted average and grayscale ratio, determine the quality parameters corresponding to the grayscale image group, including:

[0028] By combining the first and second weights, a weighted average and grayscale ratio are calculated, and the weighted value obtained from the weighted calculation is used as the quality parameter corresponding to the grayscale image group.

[0029] Therefore, AR glasses can adjust the degree of influence on image quality scoring by adjusting the first and second weights, which helps AR glasses to output images that maximize the gradient information of feature points, thus improving the recognition accuracy of AR glasses.

[0030] In one embodiment, when the quality parameter is less than a preset parameter threshold, the shooting parameters of the RGB camera are adjusted. The preset parameter threshold is a pre-defined quality parameter based on the image captured by the RGB camera under normal lighting conditions, including:

[0031] If the quality parameter is less than the preset parameter threshold, the first parameter value corresponding to the exposure time adjustment is added to the shooting parameters output to the RGB camera to control the RGB camera to adjust the exposure time according to the first parameter value.

[0032] Therefore, AR glasses score the image quality of the images acquired by the RGB camera, and adjust the RGB camera in a timely manner when the current shooting parameters of the RGB camera cannot adapt to the current lighting environment. This allows the AR glasses to dynamically adapt to image recognition under different lighting conditions, effectively improving the recognition accuracy of the AR glasses.

[0033] Secondly, this application also provides an image recognition optimization device for use in AR glasses. The AR glasses include an RGB camera for acquiring RGB image stream data. The image recognition optimization device includes:

[0034] The image grayscale module is configured to perform grayscale processing on multiple frames of target images in the RGB image stream data to obtain corresponding multi-frame grayscale images.

[0035] The data acquisition module is configured to confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group, wherein the grayscale image group includes multiple frames of grayscale images after grayscale processing.

[0036] The image scoring module is configured to determine the quality parameters corresponding to a group of grayscale images based on the weighted average value and the grayscale ratio.

[0037] The parameter adjustment module is configured to adjust the shooting parameters of the RGB camera when the quality parameters are less than a preset parameter threshold. The preset parameter threshold is the quality parameters obtained in advance based on the images captured by the RGB camera under normal lighting conditions.

[0038] Thirdly, this application also provides AR glasses, which include:

[0039] One or more processors;

[0040] Storage device for storing one or more programs.

[0041] When one or more programs are executed by one or more processors, the one or more processors implement the image recognition optimization method of this application.

[0042] Fourthly, this application also provides a storage medium for storing computer-executable instructions, which, when executed by a processor, are used to perform the image recognition optimization method of this application. Attached Figure Description

[0043] Figure 1 This is a schematic diagram illustrating the steps of an image recognition method provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram illustrating the steps of data processing for a group of grayscale images according to an embodiment of this application;

[0045] Figure 3 This is a schematic diagram illustrating the steps for obtaining a weighted average value according to an embodiment of this application;

[0046] Figure 4 This is a schematic diagram illustrating the steps for obtaining a grayscale ratio value according to an embodiment of this application;

[0047] Figure 5 This is a comparison diagram of image recognition effects provided in an embodiment of this application;

[0048] Figure 6 A schematic diagram of the structure of an image recognition optimization device provided in an embodiment of this application.

[0049] Figure 7 This is a schematic diagram of the structure of AR glasses provided in an embodiment of this application. Detailed Implementation

[0050] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, the accompanying drawings only show the parts related to the embodiments of this application, not all structures. Those skilled in the art, after reading this specification, should be able to conceive that any combination of technical features can constitute an optional implementation method, provided that the technical features do not contradict each other.

[0051] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this application, "multiple" means two or more, and "several" means one or more.

[0052] To promote digital displays that present cultural relics and exhibits to visitors in a more vivid and engaging way, museums will install digital equipment, such as AR glasses, next to the display cases. Visitors can wear AR glasses to view the exhibits inside the cases. AR glasses can present digital models of the exhibits, making them more lifelike and technologically advanced, thus enhancing the visitor experience.

[0053] The images captured by the RGB camera in AR glasses are primarily composed of a mixture of red, green, and blue primary colors of light. Their image quality is highly dependent on the ambient light, meaning AR glasses only achieve good recognition rates under normal lighting conditions. For example, in museums or exhibition halls, the environment is typically low-light, while the exhibits in the display cases create a stark contrast with their surroundings. If illuminated by strong light, the areas around the exhibits become brightly lit. However, in these low- or high-brightness background lighting environments, RGB images tend to be either too dark or overexposed, resulting in poor recognition accuracy for the AR glasses and making it difficult to provide visitors with accurate digital models.

[0054] Therefore, this application provides an image recognition optimization method. Figure 1 This diagram illustrates the steps of an image recognition optimization method provided in an embodiment of this application. This method can be applied to AR glasses. It is conceivable that AR glasses acquire RGB image stream data through their RGB cameras. In the AR glasses, the quality of the images in the RGB image stream data is scored to determine the corresponding quality parameters. Based on these quality parameters, the shooting parameters of the RGB camera are adjusted so that the RGB camera can acquire better quality images, thereby improving the recognition accuracy of the AR glasses. Accordingly, the specific steps are as follows:

[0055] Step S110: Perform grayscale processing on the multiple target images in the RGB image stream data to obtain the corresponding multiple grayscale images.

[0056] Understandably, when the processor in the AR glasses receives RGB image stream data, it performs grayscale processing on multiple target images in the RGB image. The target image is a number of frames in the image captured within a certain period of time. For example, if the RGB camera captures RGB image stream data at an image acquisition rate of 24 frames per second, that is, the RGB camera captures 24 frames of images in one second, the processor in the AR glasses can select the first 10 frames or the last 10 frames as the target image.

[0057] After acquiring the target image, the AR glasses' processor performs grayscale processing on the target image, converting it into a grayscale image. For example, during the grayscale processing, the processor can use preset grayscale processing methods such as the maximum value method, the average value method, or the weighted average method to convert the components of different color channels in the target image into grayscale values. It is conceivable that, since there are multiple target images, a corresponding number of grayscale images can be obtained after grayscale processing.

[0058] Step S120: Confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group. The grayscale image group includes multiple grayscale images after grayscale processing.

[0059] Step S130: Determine the quality parameters corresponding to the grayscale image group based on the weighted average value and the grayscale ratio.

[0060] A grayscale image set comprises multiple frames of grayscale images that have undergone grayscale conversion. The AR glasses require further processing of each frame within the set. For example, the AR glasses extract feature points from each frame to obtain a weighted average of these feature points. Additionally, they perform grayscale statistics on each frame to obtain the corresponding grayscale ratio. Based on the obtained weighted average and grayscale ratio, the AR glasses can assign a quality score to the grayscale image set to determine its corresponding quality parameters.

[0061] Step S140: When the quality parameter is less than the preset parameter threshold, adjust the shooting parameters of the RGB camera. The preset parameter threshold is the quality parameter corresponding to the image captured by the RGB camera under normal lighting conditions.

[0062] In addition, the AR glasses have preset parameter thresholds. These thresholds are essentially quality parameters derived from images captured by the RGB camera under normal lighting conditions. When the quality parameters are below the preset thresholds, the AR glasses adjust the RGB camera's shooting parameters to adapt to the current lighting environment, reducing the likelihood of overexposed or underexposed images. This results in higher-quality images for image recognition, providing users with better image recognition performance.

[0063] It is conceivable that in this round of testing, if the AR glasses determine that the current quality parameter is greater than or equal to the preset parameter threshold, the AR glasses will run the preset image recognition algorithm to perform image recognition on the acquired image, and then display the processed image to the user.

[0064] Therefore, AR glasses perform grayscale processing on the acquired image to obtain the grayscale ratio and the weighted average of the corresponding image feature points, thus determining the corresponding quality parameters. If the quality parameters are lower than a preset threshold, it can be determined that the current RGB camera is not adapted to the current lighting environment, and the shooting parameters of the RGB camera can be adjusted to suit the lighting conditions. In this way, AR glasses can dynamically adapt to different lighting environments by scoring the acquired image quality and adjusting the RGB camera when the image quality does not meet the requirements. This allows the AR glasses to output images that maximize the gradient information of feature points, thereby improving the accuracy of image recognition under different lighting conditions and providing users with better image recognition results.

[0065] In some embodiments, such as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the steps of data processing for a group of grayscale images according to an embodiment of this application. The AR glasses extract feature points and perform grayscale statistics on each frame of grayscale images within the grayscale image group. It should be understood that the feature point extraction and grayscale statistics processes described above can be performed sequentially or simultaneously by the AR glasses. The specific steps are as follows:

[0066] Step S210: Extract feature points for each frame of grayscale image, obtain the image feature points and grayscale values ​​of the image feature points for each frame of grayscale image, and determine the weighted average value of the grayscale image group.

[0067] Step S220: Perform grayscale statistics on each frame of grayscale image to obtain the grayscale ratio value of the grayscale image group.

[0068] Understandably, for each frame within a grayscale image group, the AR glasses extract feature points from each frame to obtain the image feature points and the grayscale values ​​of the corresponding pixels. After determining the image feature points for each frame, the AR glasses can determine a weighted average value corresponding to the image feature points based on these feature points and their corresponding pixel values. For example, they can first determine the average grayscale value of all image feature points in each grayscale image, and then calculate the weighted average value using this average value.

[0069] In addition, for the grayscale statistics of each frame of grayscale image, AR glasses can perform grayscale value statistics on all pixels in each acquired grayscale image, thereby determining the proportion of each grayscale value pixel in all images in the grayscale image group, and thus obtaining the corresponding grayscale ratio value.

[0070] Therefore, by extracting feature points from each grayscale image within a grayscale image group and performing grayscale statistics, AR glasses can determine the image quality of the grayscale image group, and then adjust the RGB camera to adapt to the current lighting environment, thereby helping to improve the image recognition accuracy of AR glasses and making the image recognition effect of AR glasses better.

[0071] In one embodiment, the AR glasses need to extract feature points from each frame of a grayscale image group to obtain a weighted average of the feature points corresponding to the grayscale image group. For example... Figure 3 As shown, Figure 3 A schematic diagram illustrating the steps for obtaining a weighted average value according to an embodiment of this application is shown below.

[0072] Step S310: According to the preset feature point detection algorithm, feature points are extracted from all grayscale images to determine the image feature points on the grayscale images, and the grayscale values ​​corresponding to the image feature points are used as the feature point values ​​of the image feature points.

[0073] Step S320: Average the feature point values ​​of all image feature points on each frame of grayscale image to determine the average feature point value of each frame of grayscale image.

[0074] Step S330: Based on the weighting coefficients corresponding to each grayscale image, perform a weighted average calculation on the average value of feature points of all grayscale images to determine the weighted average value of the grayscale image group.

[0075] Understandably, AR glasses are pre-configured with corresponding feature point detection algorithms to detect image feature points. For example, the FAST (Features from Accelerated Segment Test) algorithm can be used as a pre-configured feature point detection algorithm to detect image feature points on each grayscale image; that is, feature point extraction is performed on each grayscale image using the FAST algorithm. Then, after determining the image feature points, the grayscale value of the pixel corresponding to the image feature point is used as the feature point value of that image feature point.

[0076] AR glasses can calculate the average feature point value of each grayscale image frame based on the number of image feature points and the corresponding feature point values. This involves averaging the feature point values ​​of all image feature points in each grayscale image frame to determine the average feature point value. For example, by counting the feature point values ​​of all image feature points in each grayscale image frame and averaging the statistical values ​​obtained from the count of feature points in that frame, the average feature point value for that frame is determined. Similarly, this averaging process is performed on other grayscale images to obtain the average feature point value for each grayscale image.

[0077] It's conceivable that AR glasses would have pre-set weighting coefficients to correspond to each frame of grayscale image. For example, an RGB camera might capture 24 frames per second, and the processor in the AR glasses would use the first 10 frames as the target images. Ten corresponding weighting coefficients could be pre-set within the AR glasses to correspond one-to-one with the grayscale images after grayscale processing. After determining the weighting coefficients, a weighted average of the feature points of all grayscale images would be calculated by combining the weighting coefficients corresponding to each grayscale image, thus obtaining the weighted average of the grayscale image group.

[0078] Therefore, AR glasses calculate the weighted average of grayscale image groups to obtain the quality score of the images acquired by the RGB camera. This allows the AR glasses to adjust the RGB camera in a timely manner, enabling the RGB camera to dynamically adapt to the image recognition of AR glasses under different lighting conditions, which is beneficial to improving the recognition accuracy of AR glasses.

[0079] In one embodiment, after completing the grayscale processing of the image, the AR glasses will also perform grayscale statistics on the acquired grayscale image to obtain the corresponding grayscale statistical results, and then determine the grayscale ratio value of the current grayscale image group. For example... Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the steps for obtaining the grayscale ratio value according to an embodiment of this application. The specific steps are as follows:

[0080] Step S410: Based on the grayscale statistical results, determine the grayscale values ​​corresponding to all pixels in each frame of grayscale image.

[0081] Step S420: Determine the average grayscale value of each grayscale image based on the grayscale value of all pixels in each grayscale image and the number of pixels.

[0082] Step S430: Based on the weighting coefficients corresponding to each grayscale image, perform a weighted average operation on the grayscale average values ​​corresponding to all grayscale images to determine the grayscale ratio of the grayscale image group.

[0083] It is understandable that for each frame of grayscale image within a grayscale image group, AR glasses need to perform further grayscale statistics to obtain grayscale statistics results. In one embodiment, the grayscale statistics results can be grayscale histograms to obtain the grayscale values ​​corresponding to all pixels in each frame of grayscale image, and also to determine the number of pixels at each grayscale value, thereby helping to improve the efficiency of grayscale statistics.

[0084] Accordingly, the gray values ​​of each pixel in each frame of grayscale image are averaged, that is, the sum of the gray values ​​of each pixel in each frame of grayscale image is calculated, and then the average gray value of the grayscale image is calculated based on the number of pixels in the frame of grayscale image.

[0085] Furthermore, after obtaining the average grayscale value corresponding to each frame of grayscale image, the AR glasses also perform a weighted average calculation on the obtained average grayscale value corresponding to each frame of grayscale image based on the weighting coefficient corresponding to each frame of grayscale image, to obtain the grayscale ratio value of the grayscale image group. It can be conceivable that the weighting coefficients used in calculating the grayscale ratio value and the weighted average value corresponding to the image feature points can both use the same value.

[0086] Therefore, AR glasses calculate the grayscale ratio of grayscale image groups to obtain the quality score of the image acquired by the RGB camera. This allows the AR glasses to adjust the RGB camera in a timely manner, enabling the RGB camera to dynamically adapt to the image recognition of AR glasses under different lighting conditions, which is beneficial to improving the recognition accuracy of AR glasses.

[0087] In some embodiments, for calculating the quality parameters of a grayscale image group, the AR glasses can combine a preset first weight and a second weight to perform a weighted operation on the weighted average and the grayscale ratio value, and then use the weighted value obtained from the weighted operation as the quality parameter corresponding to the grayscale image group. The first weight corresponds to the weighted average, the second weight corresponds to the grayscale ratio value, and the sum of the first weight and the second weight is 1. The specific formula is as follows:

[0088] {ImageQuality}=w1*{cur_keypoints}+w2*{gray_values}

[0089] w1 + w2 = 1

[0090] Where {ImagerQuality} is the quality parameter, w1 is the first weight, w2 is the second weight, {cur_keypoints} is the weighted average, and {gray_values} is the grayscale ratio.

[0091] Understandably, the first and second weights are used to adjust the influence of the weighted average value corresponding to image feature points and the grayscale ratio on image quality. Accordingly, the values ​​of the first and second weights can be selected based on the design requirements of the AR glasses. Therefore, by adjusting the first and second weights, the AR glasses can regulate their influence on image quality scoring, thereby helping them to better output images that maximize the gradient information of feature points, thus improving the recognition accuracy of the AR glasses.

[0092] In some embodiments, when the quality parameter is less than a preset parameter threshold, the AR glasses further adjust the shooting parameters of the RGB camera. For example, a first parameter value corresponding to adjusting the exposure duration is added to the shooting parameters output by the RGB camera to control the RGB camera to adjust the exposure duration according to the first parameter value. It is understood that, in one embodiment, the AR glasses can adjust the exposure duration by adjusting the shutter speed of the RGB camera; that is, using a preset shutter speed as the first parameter value, the AR glasses adjust the exposure duration of the RGB camera according to this shutter speed when the quality parameter is less than the preset parameter threshold. It should be noted that, in some embodiments, a second parameter value adjusting the aperture size can also be added to the shooting parameters to adjust the amount of light entering the RGB camera or, in conjunction with the shutter speed, adjust the brightness of the image after imaging.

[0093] Therefore, AR glasses score the image quality of the images acquired by the RGB camera, and adjust the RGB camera in a timely manner when the current shooting parameters of the RGB camera cannot adapt to the current lighting environment. This allows the AR glasses to dynamically adapt to image recognition under different lighting conditions, effectively improving the recognition accuracy of the AR glasses.

[0094] For example, in museums, artifacts in display cases are typically surrounded by bright lighting to highlight them in the exhibition hall. The before-and-after images of the image recognition optimization method provided in this application during image recognition using AR glasses are shown below. Figure 5 As shown, Figure 5The image recognition effect comparison diagram provided in one embodiment of this application shows three groups of images. The left image in each group is the image obtained by the AR glasses without adjustment using the method provided in this application, and the right image is the image obtained by the AR glasses with adjustment using the method provided in this application.

[0095] In museums, artifacts in display cases are typically surrounded by bright lighting to highlight them in the exhibition space. As shown in the left-hand images of each image group in the figure, during image recognition using AR glasses on artifacts in museum display cases, the images acquired by the AR glasses are overexposed due to the bright lighting surrounding the artifacts, resulting in a loss of image detail. AR glasses exhibit poor recognition accuracy for such overexposed images.

[0096] After optimization by the solution in this application, the images acquired by the AR glasses are shown as the right images in each image group in the figure. After comparing the right images with the left images in each group, it can be clearly observed that the brightness of the right images (which can be reflected by the grayscale of the images in the figure) is reduced compared to the left images. That is, the acquired images are no longer overexposed, thus preserving the details of the images and improving the recognition accuracy of the AR glasses.

[0097] Figure 6 This is a schematic diagram of the structure of an image recognition optimization device provided in an embodiment of this application. The device is used to execute the image recognition optimization method provided in the above embodiment and has functional modules and beneficial effects corresponding to the execution method. As shown in the figure, the device includes an image grayscale conversion module 601, a data acquisition module 602, an image scoring module 603, and a parameter adjustment module 604.

[0098] The image grayscale module 601 is configured to perform grayscale processing on multiple target images in the RGB image stream data to obtain corresponding multiple grayscale images.

[0099] The data acquisition module 602 is configured to confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group. The grayscale image group includes multiple frames of grayscale images after grayscale processing.

[0100] The image scoring module 603 is configured to determine the quality parameters corresponding to the grayscale image group based on the weighted average value and the grayscale ratio value;

[0101] The parameter adjustment module 604 is configured to adjust the shooting parameters of the RGB camera when the quality parameters are less than a preset parameter threshold. The preset parameter threshold is the quality parameters obtained in advance based on the images captured by the RGB camera under normal lighting conditions.

[0102] Based on the above embodiments, the data acquisition module 602 is further configured as follows:

[0103] Feature points are extracted from each frame of grayscale image to obtain the image feature points and grayscale values ​​of each feature point, so as to determine the weighted average value of the grayscale image group.

[0104] Perform grayscale statistics on each frame of grayscale image to obtain the grayscale ratio value of the grayscale image group.

[0105] Based on the above embodiments, the data acquisition module 602 is further configured as follows:

[0106] Based on the preset feature point detection algorithm, feature points are extracted from all grayscale images to determine the image feature points on the grayscale images, and the grayscale value corresponding to the image feature point is used as the feature point value of the image feature point.

[0107] The average value of the feature points of all image feature points in each frame of grayscale image is calculated to determine the average value of the feature points of each frame of grayscale image.

[0108] Based on the weighting coefficients corresponding to each grayscale image, a weighted average is calculated on the average value of the feature points of all grayscale images to determine the weighted average value of the grayscale image group.

[0109] Based on the above embodiments, the data acquisition module 602 is further configured as follows:

[0110] Based on the grayscale statistical results, determine the grayscale value of all pixels in each frame of grayscale image;

[0111] The average grayscale value of each grayscale image is determined based on the grayscale value of all pixels in each grayscale image and the number of pixels.

[0112] Based on the weighting coefficients corresponding to each grayscale image, a weighted average is calculated on the grayscale average values ​​of all grayscale images to determine the grayscale ratio of the grayscale image group.

[0113] Based on the above embodiments, the grayscale statistical result is a grayscale histogram.

[0114] Based on the above embodiments, the first weight corresponds to the weighted average value, the second weight corresponds to the grayscale ratio value, and the sum of the first weight and the second weight is 1; the image scoring module 603 is further configured as follows:

[0115] Combining the first and second weights, a weighted average and grayscale ratio are calculated, and the weighted value obtained from the weighted calculation is used as the quality parameter corresponding to the grayscale image group.

[0116] Based on the above embodiments, the parameter adjustment module 604 is further configured as follows:

[0117] If the quality parameter is less than the preset parameter threshold, the first parameter value corresponding to the exposure time adjustment is added to the shooting parameters output to the RGB camera to control the RGB camera to adjust the exposure time according to the first parameter value.

[0118] It is worth noting that in the above-described embodiments of the image recognition optimization device, the modules are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each module are only for easy differentiation and are not used to limit the scope of protection of this application.

[0119] Figure 7 This is a schematic diagram of the structure of AR glasses provided in an embodiment of this application. The device is used to execute the image recognition optimization method provided in the above embodiment and has corresponding functional modules and beneficial effects for executing the method. As shown in the figure, the AR glasses include a processor 701, a memory 702, an input device 703, and an output device 704. The number of processors 701 can be one or more; one processor 701 is shown as an example in the figure. The processor 701, memory 702, input device 703, and output device 704 can be connected via a bus or other means; a bus connection is shown as an example in the figure. The memory 702, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the image recognition optimization method in the embodiments of this application. The processor 701 executes various corresponding functional applications and data processing by running the software programs, instructions, and modules stored in the memory 702, thereby realizing the above-mentioned image recognition optimization method.

[0120] The memory 702 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data recorded or created during use. Furthermore, the memory 702 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may further include memory remotely configured relative to the processor 701, which can be connected to a terminal device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0121] The input device 703 can be used to input corresponding digital or character information to the processor 701, and to generate key signal inputs related to the user settings and function control of the device; the output device 704 can be used to send or display key signal outputs related to the user settings and function control of the device.

[0122] This application also provides a storage medium storing computer-executable instructions, which, when executed by a processor, are used to perform related operations in the image recognition optimization method provided in any embodiment of this application.

[0123] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0125] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. An image recognition optimization method, characterized in that, Applied to AR glasses, the AR glasses including an RGB camera for acquiring RGB image stream data, the method includes: The target images in the RGB image stream data are converted to grayscale to obtain corresponding grayscale images. Confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group, wherein the grayscale image group includes the multi-frame grayscale images after grayscale processing; Based on the weighted average value and the grayscale ratio value, the quality parameter corresponding to the grayscale image group is determined, wherein the quality parameter is a weighted value obtained by weighting the weighted average value and the grayscale ratio value. If the quality parameter is less than a preset parameter threshold, the shooting parameters of the RGB camera are adjusted. The preset parameter threshold is a quality parameter obtained in advance based on the image captured by the RGB camera under normal lighting conditions. The confirmation of the weighted average value corresponding to image feature points of the grayscale image group and the grayscale ratio value of the grayscale image group includes: Feature points are extracted from each frame of the grayscale image to obtain the image feature points and grayscale values ​​of each image feature point, so as to determine the weighted average value of the grayscale image group. Perform grayscale statistics on each frame of the grayscale image to obtain the grayscale ratio value of the grayscale image group; The step of extracting feature points from each frame of the grayscale image to obtain the image feature points and grayscale values ​​of the image feature points for each frame of the grayscale image, in order to determine the weighted average value of the grayscale image group, includes: According to a preset feature point detection algorithm, feature points are extracted from all grayscale images to determine image feature points on the grayscale images, and the grayscale value corresponding to the image feature point is used as the feature point value of the image feature point. The feature point values ​​of all image feature points in each frame of the grayscale image are averaged to determine the average feature point value of each frame of the grayscale image. Based on the weighting coefficients corresponding to each grayscale image, a weighted average calculation is performed on the average value of feature points of all grayscale images to determine the weighted average value of the grayscale image group. The step of performing grayscale statistics on each frame of the grayscale image to obtain the grayscale ratio value of the grayscale image group includes: Based on the grayscale statistical results, determine the grayscale value of all pixels in each frame of the grayscale image; The average grayscale value of each frame of the grayscale image is determined based on the grayscale value of all pixels and the number of pixels in each frame of the grayscale image. Based on the weighting coefficients corresponding to each grayscale image, a weighted average calculation is performed on the grayscale average values ​​corresponding to all grayscale images to determine the grayscale ratio of the grayscale image group.

2. The image recognition optimization method according to claim 1, characterized in that, The grayscale statistical results are presented as a grayscale histogram.

3. The image recognition optimization method according to claim 1, characterized in that, The first weight corresponds to the weighted average value, the second weight corresponds to the grayscale ratio value, and the sum of the first weight and the second weight is 1; The step of determining the quality parameters corresponding to the grayscale image group based on the weighted average value and the grayscale ratio value includes: Combining the first weight and the second weight, a weighted average value and the grayscale ratio value are weighted, and the weighted value obtained by the weighted operation is used as the quality parameter corresponding to the grayscale image group.

4. The image recognition optimization method according to claim 1, characterized in that, When the quality parameter is less than a preset parameter threshold, the shooting parameters of the RGB camera are adjusted. The preset parameter threshold is a pre-defined quality parameter based on the image captured by the RGB camera under normal lighting conditions, including: If the quality parameter is less than the preset parameter threshold, a first parameter value corresponding to adjusting the exposure duration is added to the shooting parameters output to the RGB camera to control the RGB camera to adjust the exposure duration according to the first parameter value.

5. An image recognition optimization device, characterized in that, The device is applied to AR glasses, which include an RGB camera for acquiring RGB image stream data. The image grayscale module is configured to perform grayscale processing on multiple frames of target images in the RGB image stream data to obtain corresponding multi-frame grayscale images. The data acquisition module is configured to confirm the weighted average value of the grayscale image group corresponding to the image feature points and the grayscale ratio value of the grayscale image group, wherein the grayscale image group includes the multi-frame grayscale images after grayscale processing. The image scoring module is configured to determine the quality parameters corresponding to the grayscale image group based on the weighted average value and the grayscale ratio value, wherein the quality parameters are weighted values ​​obtained by weighting the weighted average value and the grayscale ratio value. The parameter adjustment module is configured to adjust the shooting parameters of the RGB camera when the quality parameter is less than a preset parameter threshold. The preset parameter threshold is a quality parameter obtained in advance based on the image captured by the RGB camera under normal lighting conditions. The data acquisition module is further configured as follows: Feature points are extracted from each frame of the grayscale image to obtain the image feature points and grayscale values ​​of each image feature point, so as to determine the weighted average value of the grayscale image group. Perform grayscale statistics on each frame of the grayscale image to obtain the grayscale ratio value of the grayscale image group; The step of extracting feature points from each frame of the grayscale image to obtain the image feature points and grayscale values ​​of the image feature points for each frame of the grayscale image, in order to determine the weighted average value of the grayscale image group, includes: According to a preset feature point detection algorithm, feature points are extracted from all grayscale images to determine image feature points on the grayscale images, and the grayscale value corresponding to the image feature point is used as the feature point value of the image feature point. The feature point values ​​of all image feature points in each frame of the grayscale image are averaged to determine the average feature point value of each frame of the grayscale image. Based on the weighting coefficients corresponding to each grayscale image, a weighted average calculation is performed on the average value of feature points of all grayscale images to determine the weighted average value of the grayscale image group. The step of performing grayscale statistics on each frame of the grayscale image to obtain the grayscale ratio value of the grayscale image group includes: Based on the grayscale statistical results, determine the grayscale value of all pixels in each frame of the grayscale image; The average grayscale value of each frame of the grayscale image is determined based on the grayscale value of all pixels and the number of pixels in each frame of the grayscale image. Based on the weighting coefficients corresponding to each grayscale image, a weighted average calculation is performed on the grayscale average values ​​corresponding to all grayscale images to determine the grayscale ratio of the grayscale image group.

6. An AR glasses, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image recognition optimization method as described in any one of claims 1-4.

7. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a processor, are used to perform the image recognition optimization method as described in any one of claims 1-4.

Images