Numeric keyboard pressing method based on machine vision

Abstract

The embodiment of the invention provides a numeric keyboard pressing method based on machine vision. The numeric keyboard pressing method comprises the steps that an initial image of a numeric keyboard area is acquired; preprocessing the initial image to reduce noise; generating an intermediate image under a stable illumination condition based on a high dynamic range imaging technology; positioning and identifying digital keys in the intermediate image by using an image identification algorithm; and controlling the mechanical arm to press the key according to the identification result. Through the scheme of the embodiment of the invention, the problem of inaccurate image recognition caused by violent light change under a high dynamic range illumination condition can be solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method for pressing a digital keyboard based on machine vision. Background Art

[0002] When pressing the keyboard through an actuator such as a manipulator, it is necessary to use an image acquisition device such as a camera to obtain the image of the keyboard, and then obtain its position information to provide the actuator with the pressing operation. However, in practical applications, this machine vision-based method has some problems, especially under high dynamic range lighting conditions, due to drastic changes in light, which may lead to inaccurate image recognition. For example, strong direct light or sudden shadow changes will greatly reduce the quality of the image captured by the camera, thereby affecting the recognition accuracy of the system and reducing the user experience. Summary of the invention

[0003] In view of this, an embodiment of the present disclosure provides a numeric keypad pressing method based on machine vision, which at least partially solves the problems existing in the prior art.

[0004] A digital keyboard pressing method based on machine vision in the present application includes:

[0005] Get the initial image of the numeric keypad area;

[0006] Preprocess the initial image to reduce noise;

[0007] Generate intermediate images under stable lighting conditions based on high dynamic range imaging technology;

[0008] Using image recognition algorithms to locate and identify digital keys in the middle image;

[0009] The robot arm is controlled to press buttons according to the recognition results.

[0010] In a specific embodiment, generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology includes:

[0011] Perform histogram equalization on the initial image to enhance the contrast;

[0012] Based on the local adaptive exposure algorithm, exposure compensation is performed on the dark and bright areas of the image respectively;

[0013] Use the median filter algorithm to remove the salt and pepper noise in the processed image;

[0014] Multiple images with different exposure intensities are fused to form a stable intermediate image, where the formula is I 融合 = (I 暗 × w暗 + I 亮 × w 亮 ) / (w 暗 + w 亮 ), I 暗 is the pixel value of the dark light image, I 亮 is the pixel value of the bright light image, w 暗 is the weight of the dark image, w 亮 is the weight of the bright light image.

[0015] In a specific implementation, performing exposure compensation on dark areas and bright areas of an image based on a local adaptive exposure algorithm includes:

[0016] Calculate the average brightness value of each local area;

[0017] Adjust the brightness of the local area based on the average brightness value. The formula is L 调整 = L 原 + α × (L 目标 -L 原 ), L 调整 is the adjusted brightness value, L 原 is the original brightness value, α is the exposure adjustment coefficient, L 目标 is the target brightness value;

[0018] Reassemble the adjusted images into a complete image;

[0019] The adjusted image is further processed using a nonlinear tone mapping algorithm to enhance the detail information.

[0020] In a specific embodiment, the step of generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology includes:

[0021] Acquire multiple images with different exposure times;

[0022] Perform HDR synthesis on multiple images;

[0023] Brightness correction based on synthesized HDR image;

[0024] Improve the contrast of the image through histogram equalization.

[0025] In a specific embodiment, the step of performing brightness correction based on the synthesized HDR image includes:

[0026] Calculate the average brightness value Lavg of the image;

[0027] Adjust the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget;

[0028] When |Lavg- Ltarget | > T, brightness adjustment is performed, where T is the set brightness difference threshold, which is used to determine whether brightness adjustment is needed;

[0029] The brightness is adjusted using the formula Bnew = Bold × Ltarget / Lavg, where Bold and Bnew represent the original brightness value and the adjusted brightness value, respectively.

[0030] In a specific embodiment, the step of adjusting the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget includes:

[0031] Divide the image into multiple sub-regions;

[0032] Calculate the average brightness value Lsub of each sub-area;

[0033] When |Lsub - Ltarget| > T, brightness adjustment within the sub-region is performed;

[0034] The formula Bsub_new = Bsub_old × Ltarget / Lsub is used to adjust the brightness of the sub-region, where Bsub_old and Bsub_new represent the original brightness value and the adjusted brightness value of the pixels in the sub-region, respectively.

[0035] In a specific embodiment, based on the difference between the average brightness value Lsub in the sub-region and the target brightness value Ltarget, the step of adjusting the brightness in the sub-region includes:

[0036] Determine the size and location of the sub-regions;

[0037] Calculate the local dynamic range Drange of each sub-region;

[0038] When |Drange-Dtarget| > D, local dynamic range adjustment is performed, where Dtarget is the set local dynamic range target value and D is the set dynamic range difference threshold;

[0039] Using the formula Perform dynamic range adjustment, where and Represent the brightest and darkest pixel values ​​in the sub-region respectively.

[0040] In a specific embodiment, the step of adjusting the dynamic range in the sub-region based on the difference between the local dynamic range Drange of the sub-region and the target local dynamic range Dtarget includes:

[0041] Calculate the brightest and darkest pixel values ​​Bmax and Bmin in the sub-region;

[0042] When Drange > Dtarget, compress the local dynamic range;

[0043] When Drange<Dtarget, the local dynamic range is expanded;

[0044] Using the formula ,in is the adjustment factor, calculated based on the ratio of Dtarget and Drange.

[0045] The disclosed embodiment provides a method for pressing a numeric keypad based on machine vision, including: obtaining an initial image of a numeric keypad area; preprocessing the initial image to reduce noise; generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology; locating and identifying numeric keys in the intermediate image using an image recognition algorithm; and controlling a mechanical arm to press keys according to the recognition result. The solution of the disclosed embodiment can solve the problem of inaccurate image recognition caused by drastic light changes under high dynamic range lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0047] Figure 1 It is a flow chart of a method for pressing a digital keyboard based on machine vision;

[0048] Figure 2 It is a flow chart of generating intermediate images under stable lighting conditions based on high dynamic range imaging technology;

[0049] Figure 3 It is a flow chart for performing exposure compensation for dark and bright areas of an image based on a local adaptive exposure algorithm;

[0050] Figure 4 is a flow chart of steps for generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology;

[0051] Figure 5 is a flow chart of steps for performing brightness correction based on a synthesized HDR image;

[0052] Figure 6is a flow chart of steps for adjusting the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget;

[0053] Figure 7 is a flow chart of steps for adjusting brightness within a sub-region based on a difference between an average brightness value Lsub within the sub-region and a target brightness value Ltarget;

[0054] Figure 8 is a flow chart of steps for adjusting the dynamic range within a sub-region based on a difference between a local dynamic range Drange of the sub-region and a target local dynamic range Dtarget. DETAILED DESCRIPTION

[0055] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0056] Next, referring to the accompanying drawings, a method for pressing a numeric keypad based on machine vision according to the present invention is described.

[0057] First, refer to Figure 1 , describes the overall process of the digital keyboard pressing method based on machine vision of the present invention, such as Figure 1 As shown, the method includes:

[0058] S101: Acquire an initial image of the numeric keyboard area;

[0059] S102: preprocessing the initial image to reduce noise;

[0060] S103: Generate an intermediate image under stable lighting conditions based on high dynamic range imaging technology;

[0061] S104: using an image recognition algorithm to locate and recognize the digital keys in the intermediate image;

[0062] S105: Control the robot arm to press a button according to the recognition result.

[0063] In the present application, obtaining the initial image of the numeric keypad area is the first step of the entire method. In this step, the system captures the original image of the numeric keypad within the current viewing angle through a camera or visual sensor installed at a specific position. The core of this step is to ensure that the captured image can cover the entire target object to be identified, while also trying to avoid other unnecessary elements in the environment from entering the field of view and causing interference. In one embodiment, if the application scenario is a robot workstation in an automated factory, the camera can be fixed on the workbench so that its field of view is facing the specific area where the numeric keypad is placed, ensuring that a complete picture of the numeric keypad can be obtained every time.

[0064] Preprocessing the initial image to reduce noise means that after obtaining the original image, a series of processing operations are required to improve the image quality, thereby improving the effect of subsequent image recognition. The main tasks of this stage include but are not limited to removing high-frequency noise in the image, adjusting image contrast and brightness and other parameters, so that the boundaries of each component on the numeric keypad are more obvious and easier to be recognized by the algorithm. Specifically, when preprocessing an original image that may have a lot of background noise, you can first use the median filter method to smooth out some of the scattered noise, and then use the histogram equalization technology to increase the overall contrast of the picture, so that the key number part is more prominent, laying a good foundation for the subsequent key steps.

[0065] The purpose of generating intermediate images under stable lighting conditions based on high dynamic range imaging technology is to overcome the problems that may be encountered in image recognition under different lighting conditions. This technology combines multiple pictures with different exposures to form a high-quality picture that can clearly show details and has uniform color balance. For example, when the system detects that the ambient light is strong or varies greatly, it can continuously shoot several frames of the same scene but with different exposure times, and merge these frames into an HDR (High Dynamic Range) image to solve the problem that some parts are overexposed due to strong light and the shadows are underexposed. In this way, even in an environment with unstable lighting, the final image can be guaranteed to have good consistency and readability, greatly improving the reliability of subsequent recognition work.

[0066] Using image recognition algorithms to locate and identify the numeric keys in the intermediate image involves expertise and technology in the field of deep learning. The neural network model that has been trained with preliminary data can output the specific coordinates of each key and its corresponding numerical identifier after inputting a high-quality image. In this process, the key is to select a suitable data set and optimize the training strategy to improve the recognition accuracy and robustness of the model. Taking the convolutional neural network as an example, after building a standard sample library containing numeric keys of various shapes and sizes, data enhancement methods such as rotation and scaling can be used to expand the amount of training data, and advanced gradient descent algorithms such as Adam can be used to accelerate the convergence speed. Finally, the training results are tested through the validation set to ensure that the model has strong generalization capabilities, and can correctly identify the key positions in the face of numeric keyboard images with different backgrounds, styles, and even slight damage or dirt.

[0067] Controlling the robot arm to press keys according to the recognition results is a specific application link after the previous steps are completed. Based on the result data output by the algorithm, the action instructions of the robot arm are passed to the control system, so that it performs the corresponding movement to simulate the way humans touch the screen and accurately tap the corresponding keys. For example, after installing such equipment on a financial self-service platform, when a customer comes to handle a transfer business, the robot's visual system will quickly capture the numeric keyboard on the screen, calculate the position of each key value in real time, and control the finger-like structure to touch the target point through the drive device, thereby assisting the user to complete the operation efficiently and safely. In addition, in order to adapt to more types of device interfaces and meet higher operating precision requirements, a flexible gripping posture can be designed in combination with a force feedback device and a multi-degree-of-freedom robot arm to ensure that the ideal effect can be achieved in every click operation.

[0068] Next, refer to Figure 2 , describing the specific steps of the present invention for generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology.

[0069] S201: Perform histogram equalization on the initial image to enhance contrast. This process adjusts the image grayscale distribution to make the brightness and darkness areas in the image more uniform, thereby improving the overall contrast of the image. In one embodiment, assuming that when detecting a numeric keyboard, due to uneven ambient light, some key areas are too dim to be recognized, and histogram equalization can significantly improve the visibility of these areas.

[0070] S202: Exposure compensation is performed on the dark and bright areas of the image based on the local adaptive exposure algorithm. The local adaptive exposure algorithm automatically adjusts the exposure parameters by analyzing the brightness of the local area of ​​the image, thereby enhancing the dark details without affecting the highlight parts. For example, in the scene of a numeric keypad, the LED light on the keyboard may produce strong reflections, causing some areas to be overexposed, while the shadows between the keys are too dim. The local adaptive exposure algorithm can ensure that the details of these overexposed and underexposed areas are preserved.

[0071] S203: Use a median filter algorithm to remove salt and pepper noise from the processed image. Salt and pepper noise refers to black and white noise points that appear randomly in the image, which will interfere with subsequent image processing and recognition. Median filtering effectively removes these noise points by replacing each pixel point with the median value of its surrounding pixels. In one embodiment, there may be some noise points in the background of the numeric keyboard, and median filtering can help clean up these noise points to make the image clearer.

[0072] S204: Fusing multiple images with different exposure intensities to form a stable intermediate image. This process is achieved through the following formula:

[0073] I 融合 = (I 暗 × w 暗 + I 亮 × w 亮 ) / (w 暗 + w 亮 )

[0074] Among them, I 暗 is the pixel value of the dark light image, I 亮 is the pixel value of the bright light image, w 暗 is the weight of the dark image, w 亮 is the weight of the bright image. These weights are usually between 0 and 1, and the specific optimal value depends on the specific situation of the image. In practical applications, w can be determined by experiments first. 暗 and w 亮 The purpose of this formula is to preserve the details in the dark areas while avoiding the loss of information caused by excessive brightness, thereby generating an intermediate image with high quality under various lighting conditions.

[0075] For example, when detecting the pressing status of the numeric keypad, the system can take two or more images with different exposure intensities, and then use the above formula to fuse these images to obtain an intermediate image that can maintain high quality under different lighting conditions, thereby more accurately identifying the pressing status of the numeric keypad.

[0076] Next, refer to Figure 3 , describing the steps of performing exposure compensation for dark areas and bright areas of an image based on a local adaptive exposure algorithm of the present invention.

[0077] S301: Calculate the average brightness value of each local area. Specifically, the entire image is divided into several local areas, and the average of the pixel brightness values ​​of each area is calculated. This step is to obtain the basic lighting conditions of each local area, so as to prepare for further adjustment.

[0078] S302: Adjust the brightness of the local area based on the average brightness value. Specifically, the brightness adjustment is performed using the following formula: L 调整 = L 原 + α × (L 目标 -L 原 ), where L 调整 is the adjusted brightness value; L 原 is the original brightness value, that is, the calculated average brightness of the local area; α is the exposure adjustment coefficient, which usually ranges from 0 to 1 and can be adjusted according to the specific application scenario. The optimal value needs to be determined through experiments; L 目标 is the target brightness value that is expected to be achieved. The setting of this formula is designed to make the brightness of each local area gradually approach the target brightness value, thereby optimizing the overall quality of the image. The coefficient α controls the magnitude of each adjustment. A higher value may result in overexposure or underexposure, while a lower value requires more iterations to achieve the desired effect.

[0079] S303: Reassemble the adjusted images into a complete image. This means that it is necessary to ensure that the adjustment results of all local areas can be smoothly transitioned and spliced ​​together to form a continuous and seamless overall image. This step ensures that the final output image is visually natural and smooth without obvious block or boundary traces.

[0080] S304: Further use a nonlinear tone mapping algorithm to process the adjusted image to enhance detail information. Nonlinear tone mapping technology can improve the contrast and clarity of dark and bright parts of the image while maintaining the high dynamic range characteristics of the image, thereby making the details of the image, such as buttons and shadows on the numeric keypad, richer and more delicate. In one embodiment, when a machine vision system is used to detect the pressing of a numeric keypad, some areas may be too bright and some areas may be too dark due to uneven light sources. The use of this algorithm can effectively increase the amount of information in these local areas, making it easier for the machine to identify which keys are pressed. For example, if an area appears too bright due to reflection, the algorithm will appropriately reduce its brightness. Otherwise, it will enhance the contrast of the dark area to ensure that the outline of each key is clear enough to be visible.

[0081] Next, refer to Figure 4, describing the steps of generating an intermediate image under stable lighting conditions based on high dynamic range imaging technology according to another embodiment of the present invention.

[0082] S401: Acquire multiple images with different exposure times, which means acquiring multiple pictures from the same scene, which are taken at the same time but with different exposure times. The exposure time selection can cover a range from extremely dark to extremely bright, so as to better capture all the details in a high dynamic range scene. Generally, the selected exposure time may range from a few microseconds to several seconds. For example, in one embodiment, multiple images can be acquired from the ambient light above a numeric keypad, and these images use different exposure times of 1 / 1000 second, 1 / 250 second, and 2 seconds, respectively.

[0083] S402: Perform HDR synthesis processing on multiple images, that is, generate an HDR image by synthesizing images with different exposure times into one image. This process involves weight calculation and fusion algorithm for each pixel. Common HDR synthesis algorithms include weighted average method, synthesis method based on response curve, etc. In this process, the weight of each pixel is determined according to its exposure time and local contrast to ensure that each pixel in the synthesized image represents its most appropriate lighting state. Specifically, if the high-exposure image at a certain position has rich details, the pixels at this position will refer more to the data of the high-exposure image during HDR synthesis. For example, in the scene of a numeric keyboard, some keys may be very bright due to reflection, while other areas are relatively dark. HDR synthesis can balance the brightness differences in these areas.

[0084] S403: Brightness correction is performed based on the synthesized HDR image, with the purpose of adjusting the overall brightness level of the image to make it more consistent with the actual visual experience and reduce overexposure or underexposure. Brightness correction can be achieved through a series of nonlinear transformations, such as gamma correction or log correction. These transformations adjust the overall brightness of the image by modifying the pixel values ​​while maintaining the details of the highlights and shadows. The formula is expressed as I(x, y) = (I(x, y))^γ, where I(x, y) is the pixel value in the original image, and the γ value usually ranges from 0.5 to 2.5. The optimal value depends on the specific image and environment. γ > 1 makes the image brighter, and γ < 1 makes the image darker. For example, in the scene of a numeric keyboard, if some keys appear too dazzling due to direct strong light, the γ value can be appropriately lowered to reduce the effect of overexposure.

[0085] S404: Improving the contrast of the image through histogram equalization is aimed at further improving the visual effect of the image and enhancing the details and differentiation of the image. Histogram equalization makes the histogram more uniform by redistributing the pixel intensity values ​​in the image. This process improves the dynamic range of the image and makes both the dark and bright parts of the image easier to identify. Through such a transformation, the contrast in the image can be significantly improved. For example, in the scene of a numeric keypad, if the background light is dim, the text or symbols on the keys may be difficult to recognize. Through histogram equalization, the contrast between the keys and the background can be significantly enhanced, thereby improving the accuracy of image recognition.

[0086] In summary, through the above four steps, the intermediate image under stable lighting conditions can be effectively generated, thereby significantly improving the image recognition accuracy under high dynamic range lighting conditions and effectively solving the problem of inaccurate image recognition caused by drastic light changes.

[0087] Next, refer to Figure 5 , describing the steps of performing brightness correction based on the synthesized HDR image of the present invention.

[0088] S501: Calculate the average brightness value Lavg of the image. The purpose of this step is to obtain the overall brightness level of the current image to determine whether brightness adjustment is required. Lavg is obtained by traversing each pixel in the image and calculating the average brightness value of all pixels.

[0089] S502: Based on the difference between the average brightness value Lavg and the target brightness value Ltarget, adjust the brightness of each pixel. The target brightness value Ltarget is usually an ideal brightness value pre-set according to the actual application scenario. If Lavg is too high or too low, it is necessary to adjust the brightness of each pixel to reach Ltarget.

[0090] S503: When |Lavg- Ltarget | > T, brightness adjustment is performed, where T is the set brightness difference threshold, which is used to determine whether brightness adjustment is required. The function of threshold T is to trigger the brightness adjustment operation when there is a significant difference between the average brightness value and the target brightness value. The specific value of T depends on the application scenario and can usually be selected in the range of [5, 50], with the optimal value being about 20.

[0091] S504: Use the formula Bnew = Bold × Ltarget / Lavg to adjust the brightness, where Bold and Bnew represent the original brightness value and the adjusted brightness value respectively. The purpose of this formula is to adjust the brightness of each pixel to a new value so that it is closer to the target brightness value. By multiplying the original brightness value of each pixel by a proportional factor Ltarget / Lavg, it can be ensured that the overall brightness of the adjusted image is more uniform and close to the preset target brightness.

[0092] For example, in one embodiment, consider using a numeric keypad pressing method based on machine vision. After the camera captures the image of the numeric keypad area, it is found that the overall brightness of the image is low. The calculation results in Lavg = 80, while the preset target brightness value Ltarget = 150. Since |80 - 150| = 70 > T (assuming T = 20), the system determines that brightness adjustment is required. According to the formula Bnew = Bold × 150 / 80, the brightness value of each pixel is amplified by 1.875 times. After this processing, the overall brightness of the image is improved, making key recognition clearer, thereby improving the accuracy and reliability of key detection.

[0093] The above steps ensure that the brightness of the image can always be within an ideal range in the numeric keypad pressing method, thereby enhancing the robustness and accuracy of subsequent image processing.

[0094] Next, refer to Figure 6 , further describing the step of adjusting the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget of the present invention.

[0095] S601: Divide the entire image into multiple sub-regions. This is to process the brightness changes of different regions more finely, ensuring that the overall quality of the image is improved without causing some areas to be overexposed or too dark due to global processing.

[0096] S602: Calculate the average brightness value Lsub of each sub-region. The average brightness value can be obtained by adding the brightness values ​​of all pixels in the sub-region and then dividing it by the total number of pixels in the region. The purpose of this step is to determine the current brightness of each sub-region and provide basic data for subsequent brightness adjustment.

[0097] S603: When |Lsub - Ltarget| > T, perform brightness adjustment in the sub-region. Here, T is a preset threshold value used to determine whether the brightness value of the sub-region is close to the target brightness value Ltarget. If |Lsub - Ltarget| exceeds this threshold value T, it indicates that the brightness of the sub-region needs to be adjusted, otherwise no adjustment is made. The selection of threshold value T depends on the specific image processing requirements and can usually be determined based on experience or experiments. A typical threshold range is 0 to 25, and the optimal value depends on the actual application environment.

[0098] S604: Use the formula Bsub_new = Bsub_old × Ltarget / Lsub to adjust the brightness of the sub-region. In the formula, Bsub_old represents the original brightness value of the pixels in the sub-region, ranging from 0 to 255; Bsub_new represents the adjusted brightness value, also between 0 and 255. The meaning of this formula is to adjust the brightness value of each pixel to the product of the original brightness value, the target brightness value, and the current average brightness value. In this way, it is ensured that the average brightness value of the sub-region is as close as possible to the target brightness value Ltarget.

[0099] In one embodiment, it is assumed that in a digital keyboard pressing method based on machine vision, the keyboard image captured by the image acquisition device is too dark in some areas due to different lighting conditions. In order to improve the recognition accuracy, the system divides the image into 9 sub-areas, each of which is 100x100 pixels in size. After calculating the average brightness value of each sub-area, it is found that Lsub= 70 in one of the sub-areas, and the target brightness value Ltarget is set to 120. Since |70- 120| = 50, and the preset threshold T = 20, the brightness of this sub-area needs to be adjusted. According to the above formula, the brightness value of each pixel in the sub-area will be adjusted from Bsub_old to Bsub_old×120 / 70, so that the overall brightness of the sub-area is closer to 120, thereby improving the image quality and improving the recognition effect of keyboard keys.

[0100] Next, refer to Figure 7 , describing the steps of adjusting the brightness in a sub-region based on the difference between the average brightness value Lsub in the sub-region and the target brightness value Ltarget of the present invention.

[0101] S701: Determine the size and position of the sub-region;

[0102] S702: Calculate the local dynamic range Drange of each sub-region;

[0103] S703: When |Drange-Dtarget| > D, perform local dynamic range adjustment, where Dtarget is a set local dynamic range target value, and D is a set dynamic range difference threshold; S704: Using formula Perform dynamic range adjustment, where and Represent the brightest and darkest pixel values ​​in the sub-region respectively.

[0104] Determining the size and position of the sub-region refers to dividing the entire image into several smaller sub-regions according to the actual application requirements of the image. The size of the sub-region can be flexibly set according to the specific application scenario. For example, in the numeric keypad pressing method based on machine vision, the area corresponding to each key can be used as a sub-region, so that the brightness can be adjusted more accurately in the local range to ensure the recognition accuracy of each key.

[0105] Calculating the local dynamic range Drange of each sub-region refers to statistically analyzing the brightness distribution in each sub-region by calculating the brightest pixel value in the sub-region. The darkest pixel value The difference between the two is used to obtain the local dynamic range Drange of the sub-region. The purpose of this step is to understand the brightness variation range within the sub-region and provide a basis for subsequent dynamic range adjustment.

[0106] When |Drange-Dtarget| > D, performing local dynamic range adjustment means judging whether the local dynamic range Drange of the current sub-region is significantly different from the set target dynamic range Dtarget. If the difference exceeds the preset threshold D, dynamic range adjustment is required. The D value here is usually set to an empirical value, such as 50 or 100, depending on application requirements and environmental conditions. The purpose of this is to ensure that the image has a more consistent dynamic range in the local range, improve image quality, and thus better recognize and process the keys on the numeric keypad.

[0107] Using the formula Dynamic range adjustment is performed to calculate the new local dynamic range and adjust the brightness in the sub-region according to this value. Specifically, in this formula and Represent the brightest and darkest pixel values ​​in the sub-region respectively. The value range of these two parameters is [0, 255] (assuming the image is an 8-bit grayscale image). Dtarget is the preset target dynamic range, usually set to a constant value, such as 100. The meaning of this formula is to calculate a new local dynamic range Dnew from the existing brightness distribution, and adjust the new local dynamic range to be closer to the target dynamic range Dtarget. This can make the brightness change in the sub-region more uniform and reduce the image distortion caused by too bright or too dark.

[0108] For example, in one embodiment, assume that there is a numeric keyboard recognition system based on machine vision, and the task of the system is to accurately identify the keys pressed by the user under various lighting conditions. In a certain environment, some key areas may be too bright due to light reflection, while other areas may appear too dark due to shadows. At this time, the brightness can be adjusted according to the above steps. Specifically, the system first divides the numeric keyboard into several sub-areas, each of which corresponds to a key. Then the local dynamic range Drange of each sub-area is calculated, and it is determined whether adjustment is required. If the Drange of a sub-area is significantly greater than the set target value Dtarget, that is, |Drange- Dtarget| >D, the system will use the above formula to calculate the new local dynamic range and adjust the brightness within the sub-area. Finally, the adjusted image will have a more uniform brightness distribution, thereby improving the recognition accuracy of the keyboard keys.

[0109] Next, refer to Figure 8 , describing the steps of adjusting the dynamic range within a sub-region based on the difference between the local dynamic range Drange of the sub-region and the target local dynamic range Dtarget according to another embodiment of the present invention.

[0110] S801: Calculate the brightest and darkest pixel values ​​Bmax and Bmin in the sub-region. This step is to determine the brightness range in the sub-region and provide basic data for subsequent dynamic range adjustment. In this step, by traversing all pixel values ​​in the sub-region, the maximum value Bmax and the minimum value Bmin are found and recorded. These values ​​reflect the brightness limit in the sub-region and are the basis for dynamic range calculation.

[0111] S802: When Drange > Dtarget, compress the local dynamic range, specifically, determine the relationship between the local dynamic range Drange and the target local dynamic range Dtarget. When Drange > Dtarget, it means that the dynamic range in the sub-region is too large and needs to be compressed;

[0112] S803: When Drange<Dtarget, expand the local dynamic range. Specifically, when Drange<Dtarget, it means that the dynamic range in the sub-region is too small and needs to be expanded. The purpose of this step is to ensure that the final local dynamic range meets the target value to improve image quality or meet specific application requirements.

[0113] Specifically, if you need to compress the dynamic range, you can do so by reducing the gap between Bmax and Bmin. Conversely, if you need to expand the dynamic range, you can do so by increasing the gap between Bmax and Bmin. This step depends on the specific algorithm implementation, and usually involves nonlinear mapping or transformation of pixel values ​​within the sub-region.

[0114] S804: Using formula Perform dynamic range adjustment. In the formula, is an adjustment factor used to control the degree of compression or expansion of the dynamic range. The value of this factor usually ranges from 0 to 1, and the specific value is calculated based on the ratio of Drange and Dtarget. For example, when Drange > Dtarget, May be less than 1 to compress the dynamic range; when Drange < Dtarget, May be greater than 1 to extend the dynamic range. Specifically, The optimal value of can be determined through experiments and is generally between 0.5 and 2, depending on the specific situation.

[0115] In one embodiment, assume that there is a method for pressing a numeric keypad based on machine vision, and the goal is to clearly identify the keys in the image. There is a sub-region in the image that contains multiple keys, but in a high-light environment, the brightness difference between the background and the keys in the sub-region is large, resulting in a local dynamic range Drange = 255 that is much larger than the target local dynamic range Dtarget = 150. To this end, first calculate the brightest and darkest pixel values ​​in the sub-region, which are Bmax = 250 and Bmin = 5, respectively. Because Drange> Dtarget, the dynamic range needs to be compressed. By formula = Dtarget / Drange= 150 / 255≈0.59, get the adjustment coefficient = 0.59. The formula Dadjusted = 0.59 × (250 - 5) ≈ 147 is used to adjust the dynamic range in the sub-region to be close to the target value. In this way, the brightness difference of the keys in the image under different environments is reduced, which improves the recognition accuracy.

[0116] The method of the present invention has been described above, which particularly adopts high dynamic range imaging technology to generate intermediate images under stable lighting conditions. The present invention significantly improves the accuracy and reliability of image recognition in complex lighting environments, thereby making the numeric keyboard pressing method based on machine vision more efficient and reliable in practical applications.

[0117] The above is a preferred embodiment of the present application. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for pressing a digital keyboard based on machine vision, characterized in that: include: Get the initial image of the numeric keypad area; Preprocess the initial image to reduce noise; Generate intermediate images under stable lighting conditions based on high dynamic range imaging technology; Using image recognition algorithms to locate and identify digital keys in the middle image; The robot arm is controlled to press buttons according to the recognition results.

2. The method for pressing a digital keyboard based on machine vision according to claim 1, characterized in that: The intermediate images generated under stable lighting conditions based on high dynamic range imaging technology include: Perform histogram equalization on the initial image to enhance the contrast; Based on the local adaptive exposure algorithm, exposure compensation is performed on the dark and bright areas of the image respectively; Use the median filter algorithm to remove the salt and pepper noise in the processed image; Multiple images with different exposure intensities are fused to form a stable intermediate image, where the formula is I 融合 =(I 暗 × w 暗 + I 亮 × w 亮 ) / (w 暗 + w 亮 ), I 暗 is the pixel value of the dark light image, I 亮 is the pixel value of the bright light image, w 暗 is the weight of the dark image, w 亮 is the weight of the bright light image.

3. The method for pressing a digital keyboard based on machine vision according to claim 2, characterized in that: Based on the local adaptive exposure algorithm, exposure compensation is performed on the dark and bright areas of the image respectively, including: Calculate the average brightness value of each local area; Adjust the brightness of the local area based on the average brightness value. The formula is L 调整 = L 原 + α × (L 目标 -L 原 ), L 调整 is the adjusted brightness value, L 原 is the original brightness value, α is the exposure adjustment coefficient, L 目标 is the target brightness value; Reassemble the adjusted images into a complete image; The adjusted image is further processed using a nonlinear tone mapping algorithm to enhance the detail information.

4. The method for pressing a digital keyboard based on machine vision according to claim 1, characterized in that: The step of generating an intermediate image under stable lighting conditions based on the high dynamic range imaging technology comprises: Acquire multiple images with different exposure times; Perform HDR synthesis on multiple images; Brightness correction based on synthesized HDR image; Improve the contrast of the image through histogram equalization.

5. The method for pressing a digital keyboard based on machine vision according to claim 4, characterized in that: The step of performing brightness correction based on the synthesized HDR image comprises: Calculate the average brightness value Lavg of the image; Adjust the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget; When |Lavg- Ltarget | > T, brightness adjustment is performed, where T is the set brightness difference threshold, which is used to determine whether brightness adjustment is needed; The brightness is adjusted using the formula Bnew = Bold × Ltarget / Lavg, where Bold and Bnew represent the original brightness value and the adjusted brightness value, respectively.

6. The method for pressing a digital keyboard based on machine vision according to claim 5, characterized in that: The step of adjusting the brightness of each pixel based on the difference between the average brightness value Lavg and the target brightness value Ltarget includes: Divide the image into multiple sub-regions; Calculate the average brightness value Lsub of each sub-area; When |Lsub - Ltarget| > T, brightness adjustment within the sub-region is performed; The formula Bsub_new = Bsub_old × Ltarget / Lsub is used to adjust the brightness of the sub-region, where Bsub_old and Bsub_new represent the original brightness value and the adjusted brightness value of the pixels in the sub-region, respectively.

7. The method for pressing a digital keyboard based on machine vision according to claim 5, characterized in that: Based on the difference between the average brightness value Lsub in the sub-region and the target brightness value Ltarget, the step of adjusting the brightness in the sub-region includes: Determine the size and location of the sub-regions; Calculate the local dynamic range Drange of each sub-region; When |Drange-Dtarget| > D, local dynamic range adjustment is performed, where Dtarget is the set local dynamic range target value and D is the set dynamic range difference threshold; Using the formula Perform dynamic range adjustment, where and Represent the brightest and darkest pixel values ​​in the sub-region respectively.

8. The method for pressing a digital keyboard based on machine vision according to claim 5, characterized in that: Based on the difference between the local dynamic range Drange of the sub-region and the target local dynamic range Dtarget, the steps of adjusting the dynamic range within the sub-region include: Calculate the brightest and darkest pixel values ​​Bmax and Bmin in the sub-region; When Drange > Dtarget, compress the local dynamic range; When Drange<Dtarget, the local dynamic range is expanded; Using the formula Perform dynamic range adjustment, where is the adjustment factor, calculated based on the ratio of Dtarget and Drange.

Images