Pixel information determination method and device, ar glasses and storage medium

Abstract

The application provides a pixel information determination method and device, AR glasses and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: collecting a real-time scanning image in a current visual field range; determining a pixel range of a target object in the real-time scanning image and an included angle between a line of sight direction of the AR glasses and a vertical direction; converting the real-time scanning image into a target image based on the included angle, and determining a target region in the target image; the target image is a top view corresponding to the real-time scanning image; the target region comprises the target object; determining the number of each pixel in the target region within the pixel range and the number of each pixel in the target region. The application can improve the real-time performance and accuracy of pixel information determination.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, AR glasses, and storage medium for determining pixel information. Background Technology

[0002] Currently, pixel recognition in image processing typically involves visual analysis of images captured by drones to calculate factors such as the green coverage rate of a region, the ice cover rate of a river, and the snow cover rate of a street. Specifically, this involves color-coding some pixels in the drone-captured images to calculate the pixel percentage. For example, green pixels are labeled R:0G:255B:0. After loading all pixels in the image, each pixel is evaluated individually to obtain the percentage of green pixels, i.e., the green coverage rate.

[0003] However, firstly, in bad weather, the images captured by the drone will be darker, and the RGB values ​​of the pre-marked pixels will be inaccurate, resulting in a large error in the calculation results due to weather changes; secondly, the drone does not shoot vertically downwards, but at a certain tilt angle, so the scene in the image will appear larger from far to near, leading to a larger error in the calculation results; thirdly, for scenarios that require frequent data collection, a lot of manpower is needed to operate the equipment for data collection, resulting in poor real-time performance.

[0004] Therefore, the shortcomings of existing technologies are that the real-time performance and accuracy of pixel information determination are poor. Summary of the Invention

[0005] This invention provides a pixel information determination method, device, AR glasses, and storage medium to address the shortcomings of poor real-time performance and accuracy in pixel information determination in the prior art, thereby improving the real-time performance and accuracy of pixel information determination.

[0006] This invention provides a pixel information determination method for AR glasses, the method comprising:

[0007] Acquire real-time scan images within the current field of view;

[0008] Determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0009] Based on the included angle, the real-time scanned image is converted into a target image, and a target region in the target image is determined; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0010] Determine the number of pixels within the pixel range in the target region, and the number of pixels within the target region.

[0011] According to a pixel information determination method provided by the present invention, determining the pixel range of a target object in the real-time scanned image includes:

[0012] Based on the target direction within the current field of view, the target position of the real-time scanned image is determined; the target direction points towards the target object.

[0013] The pixel range corresponding to the target position in the real-time scanned image is determined as the pixel range of the target object in the real-time scanned image.

[0014] According to a pixel information determination method provided by the present invention, the step of converting the real-time scanned image into a target image based on the included angle includes:

[0015] Based on the pixel range, the cropped image of the real-time scanned image is determined;

[0016] The target image is obtained by performing a perspective transformation on the cropped image based on the included angle.

[0017] According to a pixel information determination method provided by the present invention, determining the cropped image of the real-time scanned image based on the pixel range includes:

[0018] Based on the pixel range, the grayscale image of the real-time scanned image is binarized to obtain a first image;

[0019] The first image is subjected to an erosion operation to remove white noise, resulting in a second image;

[0020] The second image is dilated to fill the black holes in it, resulting in the third image;

[0021] Determine the smallest rectangle that defines the maximum outline of the white region in the third image;

[0022] The real-time scan image is cropped according to the position of the smallest rectangle to obtain the cropped image of the real-time scan image.

[0023] According to a pixel information determination method provided by the present invention, the step of performing perspective transformation on the cropped image based on the included angle to obtain the target image includes:

[0024] Based on the included angle, determine the perspective transformation matrix;

[0025] The target image is obtained based on the cropped image and the perspective transformation matrix.

[0026] According to a pixel information determination method provided by the present invention, determining the target region of the target image includes:

[0027] Convex hull detection is performed on the target image to obtain the convex hull boundary;

[0028] The region within the convex hull boundary is defined as the target region of the target image.

[0029] According to a pixel information determination method provided by the present invention, the method further includes:

[0030] Determine the ratio between the number of pixels within the pixel range in the target region and the total number of pixels in the target region;

[0031] The ratio is displayed.

[0032] The present invention also provides a pixel information determination device for use in AR glasses, the device comprising:

[0033] The image acquisition module is used to acquire real-time scanned images within the current field of view;

[0034] The first determining module is used to determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0035] An image processing module is used to convert the real-time scanned image into a target image based on the included angle, and to determine a target region in the target image; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0036] The second determining module is used to determine the number of each pixel within the pixel range in the target region, and the number of each pixel in the target region.

[0037] The present invention also provides an AR glasses, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the pixel information determination method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the pixel information determination method as described above.

[0039] The pixel information determination method, apparatus, AR glasses, and storage medium provided by this invention involve: first, the AR glasses acquiring a real-time scanned image within the current field of view; then, determining the pixel range of a target object (e.g., greenery in a certain area, ice on a river, snow on a street, etc.) in the real-time scanned image, and the angle between the AR glasses' viewing direction and the vertical direction; next, converting the real-time scanned image into a top-down view corresponding to the real-time scanned image, i.e., the target image, and determining the target region (i.e., a certain area, a certain river, a certain street, etc.) in the target image, where the target region includes the target object; finally, determining the number of pixels within the pixel range in the target region, and the total number of pixels within the target region. Firstly, real-time determination of the pixel range of the target object avoids calculation errors caused by weather changes; secondly, converting the real-time scanned image into a top-down view corresponding to the real-time scanned image for subsequent calculations avoids calculation errors caused by the tilt of the shooting angle of the real-time scanned image; thirdly, real-time image acquisition and processing by the AR glasses improves the real-time performance of pixel information determination. Therefore, this invention improves the real-time performance and accuracy of pixel information determination. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the pixel information determination method provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of a grayscale image of a real-time scanned image provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the first image provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the third image provided in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of a cropped image of a real-time scanned image provided in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the target image provided in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of the target region in the target image provided in the embodiment of the present invention;

[0048] Figure 8 This is a schematic diagram of the pixel information determination device provided in an embodiment of the present invention;

[0049] Figure 9 This is a schematic diagram of the structure of the AR glasses provided in an embodiment of the present invention; Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] The following is combined with Figures 1 to 7 The pixel information determination method of the present invention is described.

[0052] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the pixel information determination method provided in an embodiment of the present invention. This method is applied to augmented reality (AR) glasses, such as… Figure 1 As shown, the method may include the following steps:

[0053] Step 101: Acquire real-time scan images within the current field of view;

[0054] Step 102: Determine the pixel range of the target object in the real-time scanned image, and the angle between the AR glasses' line of sight and the vertical direction;

[0055] Step 103: Based on the included angle, convert the real-time scanned image into a target image and determine the target region in the target image; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0056] Step 104: Determine the number of pixels within the pixel range in the target area, and the total number of pixels within the target area.

[0057] In step 101, after the staff puts on the AR glasses, they turn on the function button and the camera, and the AR glasses collect real-time scan images within the current field of view.

[0058] In step 102, the target object can refer to vegetation in a certain area, ice on a certain river, snow on a certain street, etc. The pixel range of the target object in the real-time scan image can refer to the pixel range of vegetation, ice, snow, etc. in the real-time scan image.

[0059] The line of sight in AR glasses can refer to the direction of the ray connecting the viewpoint and the center point of the observed scene. The vertical direction can refer to the direction perpendicular to the surface of the observed scene.

[0060] After acquiring a real-time scan image, the pixel range and angle can be determined in real time.

[0061] In step 103, the real-time scan image is an image taken by the AR glasses in the side view direction. Based on the included angle, the real-time scan image is converted into a top view corresponding to the real-time scan image, i.e., the target image, and the target area (i.e., a certain area, a certain river, a certain street, etc.) containing the target object is determined in the target image.

[0062] In step 104, the number of pixels within the pixel range in the target area, for example: the number of pixels within the pixel range corresponding to greenery in a certain area (i.e., the number of pixels of greenery in a certain area), the number of pixels within the pixel range corresponding to ice surface in a certain river (i.e., the number of pixels of ice surface in a certain river), the number of pixels within the pixel range corresponding to snow accumulation in a certain street (i.e., the number of pixels of snow accumulation in a certain street), and so on.

[0063] The number of pixels within the target area, such as the total number of pixels in a certain area, the total number of pixels in a certain river, the total number of pixels in a certain street, etc.

[0064] Calculate the number of pixels within the pixel range in the target region, and the total number of pixels within the target region.

[0065] This embodiment provides a pixel information determination method. First, AR glasses acquire a real-time scanned image within the current field of view. Then, the pixel range of a target object (e.g., greenery in a certain area, ice on a river, snow on a street, etc.) in the real-time scanned image, as well as the angle between the AR glasses' viewing direction and the vertical direction, are determined. Next, the real-time scanned image is converted into a top-down view corresponding to the real-time scanned image, i.e., the target image, based on the angle, and the target region (i.e., a certain area, a certain river, a certain street, etc.) within the target image is determined, including the target object. Finally, the number of pixels within the pixel range in the target region and the number of pixels within the target region are determined. Firstly, real-time determination of the pixel range of the target object avoids calculation errors caused by weather changes. Secondly, converting the real-time scanned image into a top-down view corresponding to the real-time scanned image for subsequent calculations avoids calculation errors caused by the tilt of the shooting angle of the real-time scanned image. Thirdly, real-time image acquisition and processing by the AR glasses improves the real-time performance of pixel information determination. Therefore, this embodiment can improve the real-time performance and accuracy of pixel information determination.

[0066] In one embodiment, step 102, determining the pixel range of the target object in the real-time scanned image, may include the following sub-steps:

[0067] Step 1021: Determine the target position in the real-time scanned image based on the target direction within the current field of view; the target direction points towards the target object;

[0068] Step 1022: Determine the pixel range corresponding to the target position in the real-time scanned image as the pixel range of the target object in the real-time scanned image.

[0069] In step 1021, the target direction within the current field of view can be the direction in which the staff member wearing AR glasses points towards the target object, such as pointing towards greenery or ice. The AR glasses can obtain the target direction within the current field of view through gesture recognition. This embodiment is not limited to this; the target direction within the current field of view can also be other directions pointing towards the target object.

[0070] The intersection of the target direction within the current field of view and the real-time scanned image is determined as the target location in the real-time scanned image.

[0071] In step 1022, the pixel range corresponding to the target position of the real-time scanned image can refer to the pixel range of a preset area centered on the target position of the real-time scanned image.

[0072] The pixel range corresponding to the target location in the real-time scanned image can be used as the pixel range of the target object in the real-time scanned image.

[0073] In this embodiment, the intersection of the target direction within the current field of view and the real-time scanned image can be determined as the target position of the real-time scanned image. Then, the pixel range corresponding to the target position in the real-time scanned image can be used as the pixel range of the target object in the real-time scanned image, which can improve the real-time performance and accuracy of pixel range determination.

[0074] In one embodiment, step 103, converting the real-time scanned image into a target image based on the included angle, may include the following sub-steps:

[0075] Step 1031: Determine the cropped image of the real-time scanned image based on the pixel range;

[0076] Step 1032: Perform perspective transformation on the cropped image based on the included angle to obtain the target image.

[0077] In step 1031, the cropped image of the real-time scanned image is an image including the target object that is cropped from the real-time scanned image.

[0078] The target object can be determined based on the pixel range. The image that includes the target object is then extracted from the real-time scanned image, which is the cropped image of the real-time scanned image.

[0079] In step 1032, perspective transformation projects the image onto a new viewing plane, also known as projective mapping.

[0080] Since there is an angle between the line of sight of AR glasses and the vertical direction, perspective transformation is performed on the cropped image based on this angle to obtain a target image that includes the target object.

[0081] In this embodiment, since there is an angle between the line of sight of the AR glasses and the vertical direction, the perspective transformation of the cropped image including the target object can be performed based on this angle to obtain the target image including the target object, which can avoid the calculation error caused by the tilt of the shooting angle of the real-time scanned image.

[0082] In one possible implementation, step 1031 may include the following sub-steps:

[0083] Step 10311: Based on the pixel range, binarize the grayscale image of the real-time scanned image to obtain the first image;

[0084] Step 10312: Perform an erosion operation on the first image to remove white noise and obtain the second image;

[0085] Step 10313: Dilate the second image to fill the black holes in the second image, and obtain the third image;

[0086] Step 10314: Determine the smallest rectangle of the maximum outline of the white area in the third image;

[0087] Step 10315: Capture the real-time scan image according to the position of the smallest rectangle to obtain the captured image of the real-time scan image.

[0088] In step 10311, the real-time scanned image is converted into a grayscale image, a binarization threshold is calculated based on the pixel range, and the grayscale image is binarized based on the binarization threshold to obtain the first image.

[0089] Taking the scenario of a river freezing over as an example, Figure 2 The grayscale image of the real-time scanned image of the river channel is binarized to obtain the following result: Figure 3 The first image shown. In the first image, the white areas represent the ice surface, and the black areas represent the non-ice surface.

[0090] In steps 10312 and 10313, the first image is eroded to obtain the second image, which removes white noise from the first image. The second image is then dilated to obtain the third image, which fills in the black holes in the second image.

[0091] Taking a frozen river scene as an example, performing erosion and dilation operations on the first image can render the entire river area white and the non-river area black, resulting in the following: Figure 4 The third image shown.

[0092] In step 10314, the contour of the white region in the third image is extracted to obtain the maximum contour of the white region in the third image. Then, the minimum bounding rectangle of the maximum contour of the white region in the third image is drawn.

[0093] In step 10315, the third image is compared with the real-time scanned image, and the real-time scanned image is cropped according to the position of the smallest rectangle of the largest outline of the white area in the third image, so as to obtain the cropped image of the real-time scanned image.

[0094] Taking a frozen river scene as an example, the maximum outline of the white area in the third image represents the river region in the third image, and the minimum rectangle of the maximum outline of the white area in the third image represents the minimum bounding rectangle of the river region in the third image. The real-time scan image is cropped according to the position of the minimum bounding rectangle, resulting in the following... Figure 5 The image shown is a cropped image of the real-time scan.

[0095] In this embodiment, an image of the smallest bounding rectangle of the white area can be extracted from the real-time scanned image, i.e., the cropped image.

[0096] In one possible implementation, step 1032 may include the following sub-steps:

[0097] Step 10321: Determine the perspective transformation matrix based on the included angle;

[0098] Step 10322: Obtain the target image based on the cropped image and the perspective transformation matrix.

[0099] In step 10321, the perspective transformation matrix can be calculated based on the included angle:

[0100]

[0101] Where Transform represents the perspective transformation matrix, a ij (i = 1, 2, 3, j = 1, 2, 3) represent the elements of the perspective transformation matrix. T1 represents the linear transformation matrix of the image, T2 = [a 13 a23 ], T2 represents the image perspective transformation matrix, T3 = [a 31 a 32 ], T3 represents the image translation matrix.

[0102] In step 10322, perspective transformation is performed using the following expression:

[0103]

[0104] Where (u, v) represents the pixel coordinates of the cropped image, and (x = x′ / w′, y = y′ / w′) are the pixel coordinates of the target image.

[0105] Alternatively, the warpPerspective() function provided by OpenCV can be used to perform perspective transformation on an image. The input parameters of the warpPerspective() function include: the four coordinates of the cropped image and the perspective transformation matrix, and the output is a target image of the same size as the cropped image.

[0106] Taking a river ice scene as an example, the perspective transformation matrix is ​​calculated based on the included angle, and then the perspective transformation matrix is ​​used to... Figure 5 The cropped image shown is subjected to perspective transformation to obtain... Figure 6 The target image shown.

[0107] In this embodiment, the perspective transformation matrix can be calculated based on the included angle, and the cropped image can be transformed based on the perspective transformation matrix to obtain the target image, which can avoid the calculation error caused by the tilt of the shooting angle of the real-time scanned image.

[0108] In one embodiment, step 103, determining the target region in the target image, may include the following sub-steps:

[0109] Step 1033: Perform convex hull detection based on the target image to obtain the convex hull boundary;

[0110] Step 1034: Determine the region within the convex hull boundary as the target region of the target image.

[0111] In step 1033, the convex hull detection functions provided in OpenCV include: isCon tourConvex(contours) and convexHull(points,hull,clockwise,returnPoints).

[0112] The `isContourConvex(contours)` method is used to determine whether a given contour is a convex contour, returning a boolean value. It returns `true` if the contour is convex and `false` if it is not. However, it's important to note that the contour must be a simple contour, meaning it must be either convex or concave, and cannot be self-intersecting; otherwise, the return value will be undefined.

[0113] The `convexHull(points,hull,clockwise,returnPoints)` method is used to find the convex hull. `points` represents the input set of points (the contour point set); `hull` represents the set of vertex coordinates of the obtained convex hull, and the output format is determined by the fourth parameter `returnPoints`; `clockwise` indicates whether the direction of finding the convex hull vertices is clockwise or counterclockwise, and the default parameter `false` is sufficient; `returnPoints`, if `true`, outputs the vertex set of the convex hull; if `false`, outputs a vector of integers containing the indices of each vertex of the convex hull in the contour point set.

[0114] Taking the scenario of river freezing as an example, the above arbitrary convex hull detection function can be used to detect... Figure 6 The target image shown is subjected to convex hull detection to obtain the convex hull boundary, which is the boundary of the river channel.

[0115] In step 1034, the region within the convex hull boundary (i.e., the region within the boundary of the river channel) is defined as the target region (i.e., the river channel region). Figure 7 The area within the white line shown.

[0116] In this embodiment, the target region of the target image can be obtained quickly and accurately using the convex hull detection method.

[0117] Optionally, the method further includes: determining the ratio between the number of pixels within a pixel range in the target region and the total number of pixels in the target region; and displaying the ratio.

[0118] Taking a frozen river scene as an example, the number of pixels within a pixel range in the target area can refer to the number of pixels on the ice surface of a certain river, while the number of pixels within the target area can refer to the total number of pixels in a certain river. The ratio of the two is the freezing rate of a certain river. After calculating the ratio, the AR glasses display the ratio.

[0119] When the icing rate exceeds a preset percentage, the AR glasses can issue an alarm. Alarm notification methods include, but are not limited to: the AR glasses displaying an alarm message (e.g., the alarm message can be at least one of text, images, or animations); the AR glasses triggering alarm notifications on other devices (e.g., smartphones or other smart terminals), allowing staff to be promptly notified that the pixel percentage has exceeded the limit.

[0120] In this embodiment, the real-time performance and accuracy of pixel ratio determination can be improved.

[0121] The pixel information determination device provided by the present invention is described below. The pixel information determination device described below can be referred to in correspondence with the pixel information determination method described above.

[0122] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the pixel information determination device provided in an embodiment of the present invention. This device is applied to AR glasses, such as... Figure 8 As shown, the device may include:

[0123] Image acquisition module 10 is used to acquire real-time scan images within the current field of view;

[0124] The first determining module 20 is used to determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0125] Image processing module 30 is used to convert the real-time scanned image into a target image based on the included angle, and to determine a target region in the target image; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0126] The second determining module 40 is used to determine the number of each pixel in the target region that is within the pixel range, and the number of each pixel in the target region.

[0127] Optionally, the first determining module 20 is specifically used for:

[0128] Based on the target direction within the current field of view, the target position of the real-time scanned image is determined; the target direction points towards the target object.

[0129] The pixel range corresponding to the target position in the real-time scanned image is determined as the pixel range of the target object in the real-time scanned image.

[0130] Optionally, the image processing module 30 includes:

[0131] The determining unit is used to determine the cropped image of the real-time scanned image based on the pixel range;

[0132] The transformation unit is used to perform perspective transformation on the cropped image based on the included angle to obtain the target image.

[0133] Optionally, the determining unit is specifically used for:

[0134] Based on the pixel range, the grayscale image of the real-time scanned image is binarized to obtain a first image;

[0135] The first image is subjected to an erosion operation to remove white noise, resulting in a second image;

[0136] The second image is dilated to fill the black holes in it, resulting in the third image;

[0137] Determine the smallest rectangle that defines the maximum outline of the white region in the third image;

[0138] The real-time scan image is cropped according to the position of the smallest rectangle to obtain the cropped image of the real-time scan image.

[0139] Optionally, the transformation unit is specifically used for:

[0140] Based on the included angle, determine the perspective transformation matrix;

[0141] The target image is obtained based on the cropped image and the perspective transformation matrix.

[0142] Optionally, the image processing module 30 is specifically used for:

[0143] Convex hull detection is performed on the target image to obtain the convex hull boundary;

[0144] The region within the convex hull boundary is defined as the target region of the target image.

[0145] Optionally, the device further includes:

[0146] The third determining module determines the ratio between the number of pixels within the pixel range in the target region and the total number of pixels in the target region.

[0147] Figure 9 An example is a schematic diagram of the physical structure of AR glasses, such as... Figure 9As shown, the AR glasses may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a pixel information determination method, which includes:

[0148] Acquire real-time scan images within the current field of view;

[0149] Determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0150] Based on the included angle, the real-time scanned image is converted into a target image, and a target region in the target image is determined; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0151] Determine the number of pixels within the pixel range in the target region, and the number of pixels within the target region.

[0152] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the pixel information determination method provided by the above methods, the method comprising:

[0154] Acquire real-time scan images within the current field of view;

[0155] Determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0156] Based on the included angle, the real-time scanned image is converted into a target image, and a target region in the target image is determined; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0157] Determine the number of pixels within the pixel range in the target region, and the number of pixels within the target region.

[0158] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the pixel information determination methods provided above, the method comprising:

[0159] Acquire real-time scan images within the current field of view;

[0160] Determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction;

[0161] Based on the included angle, the real-time scanned image is converted into a target image, and a target region in the target image is determined; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object;

[0162] Determine the number of pixels within the pixel range in the target region, and the number of pixels within the target region.

[0163] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining pixel information, characterized in that, The method, applied to augmented reality (AR) glasses, includes: Acquire real-time scan images within the current field of view; Determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction; Based on the included angle, the real-time scanned image is converted into a target image, and a target region in the target image is determined; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object; Determine the number of pixels within the pixel range in the target region, and the total number of pixels within the target region; Determining the pixel range of the target object in the real-time scanned image includes: Based on the target direction within the current field of view, the target position of the real-time scanned image is determined; the target direction points to the target object; the target direction is the target direction within the current field of view obtained by the AR glasses through gesture recognition of the staff wearing the AR glasses; and the target position is the intersection of the target direction within the current field of view and the real-time scanned image. The pixel range corresponding to the target position in the real-time scanned image is determined as the pixel range of the target object in the real-time scanned image; the pixel range of the target object refers to the pixel range of a preset area centered on the target position in the real-time scanned image. The step of converting the real-time scanned image into a target image based on the included angle includes: determining a cropped image of the real-time scanned image based on the pixel range; and performing a perspective transformation on the cropped image based on the included angle to obtain the target image. Determining the cropped image of the real-time scanned image based on the pixel range includes: Based on the pixel range, the grayscale image of the real-time scanned image is binarized to obtain a first image; The first image is subjected to an erosion operation to remove white noise, resulting in a second image; The second image is dilated to fill the black holes in it, resulting in the third image; Determine the smallest rectangle that defines the maximum outline of the white region in the third image; The real-time scan image is cropped according to the position of the smallest rectangle to obtain the cropped image of the real-time scan image.

2. The pixel information determination method according to claim 1, characterized in that, The step of performing perspective transformation on the cropped image based on the included angle to obtain the target image includes: Based on the included angle, determine the perspective transformation matrix; The target image is obtained based on the cropped image and the perspective transformation matrix.

3. The pixel information determination method according to any one of claims 1-2, characterized in that, Determining the target region in the target image includes: Convex hull detection is performed on the target image to obtain the convex hull boundary; The region within the convex hull boundary is defined as the target region of the target image.

4. The pixel information determination method according to any one of claims 1-2, characterized in that, The method further includes: Determine the ratio between the number of pixels within the pixel range in the target region and the total number of pixels in the target region; The ratio is displayed.

5. A pixel information determining device, characterized in that, For use in AR glasses, the device includes: The image acquisition module is used to acquire real-time scanned images within the current field of view; The first determining module is used to determine the pixel range of the target object in the real-time scanned image, and the angle between the viewing direction of the AR glasses and the vertical direction; An image processing module is used to convert the real-time scanned image into a target image based on the included angle, and to determine a target region in the target image; the target image is a top view corresponding to the real-time scanned image; the target region includes the target object; The second determining module is used to determine the number of each pixel in the target region that is within the pixel range, and the number of each pixel in the target region; The first determining module is specifically used to determine the target position of the real-time scanned image based on the target direction within the current field of view; the target direction points to the target object; the target direction is the target direction within the current field of view obtained by the AR glasses through gesture recognition of the staff wearing the AR glasses; and the target position is the intersection of the target direction within the current field of view and the real-time scanned image. The pixel range corresponding to the target position in the real-time scanned image is determined as the pixel range of the target object in the real-time scanned image; the pixel range of the target object refers to the pixel range of a preset area centered on the target position in the real-time scanned image. The image processing module includes: a determining unit, configured to determine a cropped image of the real-time scanned image based on the pixel range; and a transforming unit, configured to perform perspective transform on the cropped image based on the included angle to obtain the target image. The determining unit is specifically configured to: perform binarization processing on the grayscale image of the real-time scanned image based on the pixel range to obtain a first image; perform an erosion operation on the first image to remove white noise in the first image to obtain a second image; perform a dilation operation on the second image to fill black holes in the second image to obtain a third image; determine the minimum rectangle of the maximum outline of the white area in the third image; and crop the real-time scanned image according to the position of the minimum rectangle to obtain a cropped image of the real-time scanned image.

6. An AR glasses system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the pixel information determination method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the pixel information determination method as described in any one of claims 1 to 4.

Images