Visual intelligent calibration system and method for robot screw locking

Abstract

The invention discloses a visual intelligent calibration system and method for robot screw locking, and relates to the technical field of machine vision, and the system comprises a light intensity detection module which detects the ambient light intensity in real time and outputs a light intensity signal; the camera module collects a target area image and transmits the target area image to the control module; the light source module provides illumination; the control module adjusts exposure parameters of the camera module and light source brightness of the light source module according to the received light intensity signal; and the control module is also used for processing the image input by the camera module, acquiring screw point position information from the image, controlling the mechanical arm module to move according to the point position information, and controlling the screw locking module to lock the screw. According to the method, the brightness of the light source and the parameters of the camera are adjusted by detecting the ambient light in real time in combination with image analysis, so that the machine vision system can obtain high-quality images under different illumination conditions, and the robustness of the machine vision system and the screw locking efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision. Specifically, it is a visual intelligent calibration system and method for robot screw locking. Background Art

[0002] When the workshop produces intelligent terminal devices, the production line involves machine vision-guided robot screw locking. A robot locking machine needs to consider the reusability of different products and at different workstations, as well as the inevitable interference factors of light in the use environment. Especially in precise operations such as screw locking, inaccurate exposure or uneven light source will cause the vision system to be unable to accurately identify the screw hole position, thus affecting the accuracy and efficiency of locking. In response to the influence of different ambient light on vision, it is necessary to adjust the parameters of the camera and light source to meet the vision requirements.

[0003] In the prior art, for the visual guidance robot screw locking device in the production workshop, the machine vision system usually uses fixed exposure parameters or manual adjustment of exposure, which is difficult to adapt to complex lighting environments. In addition, the dynamic change of the light source will also cause the image quality to decline, affecting the stability of the vision system. Summary of the Invention

[0004] The purpose of the present invention is to provide a visual intelligent calibration system and method for robot screw locking, which is used to solve the problems that in the prior art, the machine vision system usually uses fixed exposure parameters or manual adjustment of exposure, and it is difficult to adapt to complex lighting environments. In addition, the dynamic change of the light source will also cause the image quality to decline, affecting the stability of the vision system.

[0005] The present invention solves the above problems through the following technical solutions:

[0006] A visual intelligent calibration system for robot screw locking includes a robotic arm module and a screw locking module, and further includes a light intensity detection module, a control module, a camera module, and a light source module, wherein:

[0007] The light intensity detection module is used to detect the ambient light intensity in real time and output a light intensity signal to the control module;

[0008] The camera module is used to collect images of the target area, transmit them to the control module, and adjust the camera parameters according to the control parameters fed back by the control module;

[0009] The light source module is used to provide illumination and adjust the illumination brightness according to the control signal of the control module;

[0010] A control module, configured to adjust the exposure parameters of the camera module and the light source brightness of the light source module according to the received light intensity signal; and further configured to process the images input by the camera module, obtain screw position information therefrom, control the robotic arm module to move according to the position information, and control the screw locking module to lock the screws.

[0011] Further, the control module includes:

[0012] A light source brightness dynamic calculation unit, configured to calculate the light intensity output constraint P according to formula (1) LED :

[0013]

[0014] wherein, I tar is the target light intensity, with the unit of klux; I act is the current ambient light intensity; P set is the initial set brightness percentage of the light source; K p is the anti-oscillation proportionality coefficient; P LED ∈[0%, 100%];

[0015] A camera exposure time calculation unit, configured to calculate the camera signal-to-noise ratio constraint T according to formula (2) exp :

[0016]

[0017] wherein, C is the camera sensitivity constant, with the unit of μs·klux 0.5 ; T max is the maximum exposure time of the camera, with the unit of ms; SNR tar is the target signal-to-noise ratio; G is the constraint condition, and the calculation formula is as follows:

[0018]

[0019] wherein, G set is the preset gain; T min is the minimum exposure time of the camera, with the unit of ms;

[0020] A camera image processing unit, configured to perform image processing on the target area image received from the camera module to obtain the actual signal-to-noise ratio SNR of the current image ROI area act and screw position information;

[0021] A control unit, configured to receive the light intensity signal from the light intensity detection module and convert it into the current ambient light intensity I act , and input it into the light source brightness dynamic calculation unit and the camera exposure time calculation unit; configured to output the calculated light intensity constraint P LEDSend it to the light source module, and send the calculated camera signal-to-noise ratio constraint T exp and the constraint condition G to the camera module; it is also used to control the movement of the robotic arm module according to the screw position information received from the camera image processing unit, and control the screw locking module to perform screw locking.

[0022] Further, when the constraint condition G exceeds a preset threshold, the camera exposure time calculation unit forcibly reduces the target signal-to-noise ratio SNR tar and recalculates the constraint condition G and the camera signal-to-noise ratio constraint T exp .

[0023] Further, the control unit is also used to obtain the signal-to-noise ratio constraint feedback value T from the hardware register of the light source module exp_read , and check that the error satisfies |T exp -T exp_read |≤0.5%, otherwise reissue the parameters. If it is satisfied, continue to judge. If |SNR act -SNR tar |>2dB, trigger the micro-compensation algorithm to iteratively adjust the target signal-to-noise ratio SNR of the current image ROI area with a set step size tar for a set number of times.

[0024] A visual intelligent calibration method for robotic screw locking, including:

[0025] Step S1, continuously detect the current ambient light intensity and collect an image of the target area through the camera; when the image quality collected by the camera is unqualified, enter Step S2; otherwise, perform image processing to obtain the screw position information and enter Step S3;

[0026] Step S2, adjust the camera exposure parameters and the light source brightness according to the current ambient light intensity and preset parameters, and return to Step S1;

[0027] Step S3, control the robotic arm to move according to the screw position information, and control the screw locking module to complete the screw locking operation.

[0028] Further, the adjusting the camera exposure parameters and the light source brightness according to the current ambient light intensity and preset parameters includes:

[0029] Calculate the light intensity output constraint P according to the preset target light intensity I tar , the light source initial setting brightness percentage P set , the anti-oscillation proportionality coefficient K p and the current ambient light intensity I act : LED :

[0030]

[0031] Among them, the target light intensity I tar is in klux; P LED ∈[0%, 100%];

[0032] According to the preset maximum exposure time T of the camera max , the camera sensitivity constant C, the target signal-to-noise ratio SNR tar and the current ambient light intensity I act , and the constraint condition G, calculate the camera signal-to-noise ratio constraint T exp :

[0033]

[0034] Among them, the unit of the camera sensitivity constant C is μs·klux 0.5 ; the maximum exposure time T of the camera max is in ms; the calculation formula of the constraint condition G is as follows:

[0035]

[0036] Among them, G set is the preset gain; T min is the minimum exposure time of the camera, in ms;

[0037] Output the calculated light intensity output constraint P LED to the light source module, and send the calculated camera signal-to-noise ratio constraint T exp , and the constraint condition G to the camera module.

[0038] Furthermore, it also includes: when the constraint condition G exceeds the preset threshold, forcibly reduce the target signal-to-noise ratio SNR tar and recalculate the constraint condition G and the camera signal-to-noise ratio constraint T exp .

[0039] Furthermore, it also includes:

[0040] Obtain the signal-to-noise ratio constraint feedback value T from the hardware register of the light source module exp_read , and check that the error satisfies |T exp -T exp_read |≤0.5%, otherwise reissue the parameters. If it is satisfied, continue to judge. If |SNR act -SNR tar |>2dB, trigger the micro-compensation algorithm to iteratively adjust SNR act for a set number of times.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] The present invention adjusts the light source brightness and camera parameters by real-time detecting ambient light and combining image analysis, ensuring that the machine vision system can obtain high-quality images under different lighting conditions, thereby improving the robustness of the machine vision system and the efficiency of screw locking. Description of the Drawings

[0043] Figure 1 is the system principle block diagram of the present invention;

[0044] Figure 2 is the flow chart of the present invention;

[0045] Figure 3 is the flow chart of dynamically adjusting parameters. Detailed Embodiment

[0046] The present invention will be further described in detail below in conjunction with embodiments, but the embodiments of the present invention are not limited thereto.

[0047] Embodiment:

[0048] Combined with the attached Figure 1 As shown, a visual intelligent calibration system for robot screw locking includes a robotic arm module and a screw locking module, and further includes a light intensity detection module, a control module, a camera module, and a light source module, wherein:

[0049] The light intensity detection module is used to detect the ambient light intensity in real time and output a light intensity signal to the control module; in this embodiment, a photosensitive sensor is used to detect the light intensity;

[0050] The camera module is used to collect images of the target area, transmit them to the control module, and adjust the camera parameters according to the control parameters fed back by the control module; the camera module supports adjusting parameters such as exposure;

[0051] The light source module is used to provide illumination and adjust the illumination brightness according to the control signal of the control module; the adjustable brightness of the light source module ensures uniform illumination in the target area;

[0052] The control module is used to adjust the exposure parameters of the camera module and the light source brightness of the light source module according to the received light intensity signal; it is also used to process the images input by the camera module, obtain screw position information therefrom, control the robotic arm module to move according to the position information, and control the screw locking module to perform screw locking.

[0053] The control module can be an industrial control computer (IPC), a programmable logic controller, and the IPC integrates a visualization operating system and an image processing algorithm module.

[0054] Further, the control module includes:

[0055] A light source brightness dynamic calculation unit for calculating the light intensity output constraint P according to formula (1) LED :[[]]

[0056]

[0057] wherein, I tar is the target light intensity, with the unit of klux; I act is the current ambient light intensity; P set is the percentage of the initial set brightness of the light source; K p is the anti-oscillation proportionality coefficient; P LED ∈ [0%, 100%];

[0058] A camera exposure time calculation unit for calculating the camera signal-to-noise ratio constraint T according to formula (2) exp :[[]]

[0059]

[0060] wherein, C is the camera sensitivity constant, with the unit of μs·klux 0.5 ; T max is the maximum exposure time of the camera, with the unit of ms; SNR tar is the target signal-to-noise ratio; G is the constraint condition, and the calculation formula is as follows:

[0061]

[0062] wherein, G set is the preset gain; T min is the minimum exposure time of the camera, with the unit of ms;

[0063] A camera image processing unit for performing image processing on the target area image received from the camera module to obtain the actual signal-to-noise ratio SNR act of the current image ROI area and the screw position information;

[0064] A control unit for receiving the light intensity signal from the light intensity detection module and converting it into the current ambient light intensity I act , and inputting it into the light source brightness dynamic calculation unit and the camera exposure time calculation unit; for sending the calculated light intensity output constraint P LED to the light source module, sending the calculated camera signal-to-noise ratio constraint T exp , and the constraint condition G to the camera module; and also for controlling the movement of the robotic arm module according to the screw position information received from the camera image processing unit, and controlling the screw locking module to perform screw locking.

[0065] By adopting a closed-loop proportional regulation algorithm, the present invention preferentially regulates the brightness of the light source to suppress sudden changes in light intensity; by dynamically adjusting the camera parameters based on the signal-to-noise ratio constraint algorithm and the signal-to-noise ratio constraint algorithm, it ensures that the machine vision system can obtain high-quality images under different lighting conditions, thereby improving the accuracy and efficiency of screw locking.

[0066] Further, when the constraint condition G exceeds a preset threshold, the camera exposure time calculation unit forcibly reduces the target signal-to-noise ratio SNR tar and recalculates the constraint condition G and the camera signal-to-noise ratio constraint T exp .

[0067] Further, the control unit is further configured to obtain the signal-to-noise ratio constraint feedback value T from the light source module hardware register exp_read , and check the error range |T exp -T exp_read |≤0.5%. If it exceeds the range, the parameters are redownloaded. If |SNR act -SNR tar |>2dB, the micro-compensation algorithm is triggered (iteratively adjusting SNR tar in steps of ±1% at most 3 times).

[0068] Embodiment 2:

[0069] Combined with Figure 2 and Figure 3 as shown, a visual intelligent calibration method for robot screw locking includes an industrial control computer, a programmable logic controller, a six-axis robotic arm, an industrial camera, a photosensitive sensor, a light source module, and a screw locking module. The control module includes an industrial control computer (industrial control computer) and a programmable controller, wherein the industrial control computer integrates a visualization operating system and an image processing algorithm module.

[0070] Step 1: Initialize and connect to the hardware:

[0071] Load preset parameters:

[0072] Target light intensity I tar (unit: klux), target signal-to-noise ratio SNR tar (unit: dB), maximum exposure time T max (unit: ms), minimum exposure time T min (unit: ms)

[0073] Establish a real-time communication link between the light source module (including the light source controller) and the camera module (industrial camera) to ensure that the instruction transmission delay ≤0.5ms. Detect the execution flow of the control program

[0074] Step 2: Dynamically collect ambient light intensity

[0075] Obtain the current ambient light intensity I through a multi-channel photosensitive sensor act , with a sampling frequency ≥ 1 kHz;

[0076] Parallelly extract the actual signal-to-noise ratio SNR of the current image ROI area collected by the camera act . Determine whether the image quality is qualified. If it is qualified, obtain the screw position information through image processing, guide the robotic arm to move, and perform screw locking by the screwdriver / screw locking module. If the image quality is unqualified, determine whether the number of unqualified times exceeds the set number, such as 3 times. If it exceeds, output an alarm and wait for manual processing. Otherwise, the control program analyzes the image preprocessing result and the light intensity and enters the next step.

[0077] Step 3: Parameter collaborative calculation (executed in sequence)

[0078] 1. Light source brightness adjustment

[0079] Adopt a closed-loop proportional adjustment algorithm:

[0080]

[0081] Parameter description:

[0082] P set : Initial set brightness of the light source (percentage, default 70%)

[0083] K p : Anti-oscillation proportional coefficient (value range 0.2 - 0.3, preferably 0.25)

[0084] Output constraint: P LED ∈[0%, 100%]

[0085] 2. Exposure time calculation

[0086] Based on the signal-to-noise ratio constraint algorithm:

[0087]

[0088] Parameter description:

[0089] C: Camera sensitivity constant (unit: μs·klux^0.5, calibrated value 120) T max : Maximum exposure time to prevent motion blur (unit: ms, default 30 ms) 3. Based on the signal-to-noise ratio constraint algorithm

[0090] Execute according to the conditional branch:

[0091]

[0092] Parameter description:

[0093] G set : Preset gain value (default 1.0x)

[0094] Constraint: G ≤ 3.0x, when exceeded, force to reduce SNR tar And recalculate Step 4: Parameter distribution and closed-loop verification

[0095] Synchronously distribute P LED , T exp , G to the hardware, and adjust the parameters of the light source module and camera module. Return to Step 1.

[0096] The present invention further includes a verification mechanism:

[0097] Read the feedback value of the hardware register and check the error range (|T exp - T exp_read | ≤ 0.5%). If the error exceeds the range, re-distribute the parameters. If |SNR act - SNR tar | > 2 dB, trigger the micro-compensation algorithm (iteratively adjust SNR tar , with a step size of ±1%, up to 3 times).

[0098] By adopting a closed-loop proportional regulation algorithm, the present invention preferentially adjusts the light source brightness to suppress sudden changes in light intensity; through dynamic adjustment of camera parameters based on the signal-to-noise ratio constraint algorithm and the signal-to-noise ratio constraint algorithm, it ensures that the machine vision system can obtain high-quality images under different lighting conditions, thereby improving the accuracy and efficiency of screw locking.

[0099] Although the present invention has been described herein with reference to its illustrative embodiments, the above embodiments are only the preferred embodiments of the present invention, and the embodiments of the present invention are not limited by the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A visual intelligent calibration system for robot screw fastening, comprising a robot arm module and a screw fastening module, characterized in that: It also includes a light intensity detection module, a control module, a camera module and a light source module, wherein: A light intensity detection module, used to detect the ambient light intensity in real time and output a light intensity signal to the control module; The camera module is used to collect images of the target area and transmit them to the control module, and adjust the camera parameters according to the control parameters fed back by the control module; A light source module, used for providing illumination and adjusting the light brightness according to the control signal of the control module; The control module is used to adjust the exposure parameters of the camera module and the light source brightness of the light source module according to the received light intensity signal; it is also used to process the image input by the camera module, obtain the screw point information therefrom, and control the mechanical arm module to move according to the point information, and control the screw locking module to perform screw locking.

2. According to claim 1, a visual intelligent calibration system for robot screw locking is characterized in that: The control module comprises: The light source brightness dynamic calculation unit is used to calculate the light intensity output constraint P according to formula (1) LED : Among them, I tar is the target light intensity, in klux; I act is the current ambient light intensity; P set Initially set the brightness percentage for the light source; K p is the anti-oscillation proportional coefficient; P LED ∈[0%,100%]; The camera exposure time calculation unit is used to calculate the camera signal-to-noise ratio constraint T according to formula (2) exp : Where C is the camera sensitivity constant, in μs·klux 0.5 ; T max is the maximum exposure time of the camera, in ms; SNR tar is the target signal-to-noise ratio; G is the constraint condition, and the calculation formula is as follows: Among them, G set is the preset gain; T min The minimum exposure time of the camera, in ms; The camera image processing unit is used to process the target area image received from the camera module to obtain the actual signal-to-noise ratio (SNR) of the current image ROI area. act and screw point information; The control unit is used to receive the light intensity signal from the light intensity detection module and convert it into the current ambient light intensity I act , and input into the light source brightness dynamic calculation unit and the camera exposure time calculation unit; used to constrain the calculated light intensity output P LED Send it to the light source module and constrain the calculated camera signal-to-noise ratio T exp , constraint condition G is sent to the camera module; and is also used to control the movement of the robotic arm module according to the screw point information received from the camera image processing unit, and to control the screw locking module to perform screw locking.

3. The visual intelligent calibration system for robot screw locking according to claim 2 is characterized in that: The camera exposure time calculation unit forces the target signal-to-noise ratio SNR to be reduced when the constraint condition G exceeds a preset threshold. tar And recalculate the constraint G and the camera signal-to-noise ratio constraint T exp .

4. A visual intelligent calibration system for robot screw locking according to claim 2 or 3, characterized in that: The control unit is further configured to obtain a signal-to-noise ratio constraint return value T from the light source module hardware register. exp_read , and check that the error satisfies |T exp -T exp_read |≤0.5%, otherwise re-issue the parameters. If satisfied, continue to judge. If |SNR act -SNR tar |>2dB, the micro-compensation algorithm is triggered to iteratively adjust the target signal-to-noise ratio (SNR) of the current image ROI area with the set step size. tar Set the number of times.

5. A visual intelligent calibration method for robot screw locking, characterized in that: include: Step S1, real-time detection of the current ambient light intensity and acquisition of the target area image by the camera; when the image quality acquired by the camera is unqualified, proceed to step S2; Otherwise, perform image processing to obtain the screw position information and proceed to step S3; Step S2, adjusting the camera exposure parameters and the light source brightness according to the current ambient light intensity and the preset parameters, and returning to step S1; Step S3, controlling the robot arm to move according to the screw position information, and controlling the screw locking module to complete the screw locking operation.

6. The visual intelligent calibration method for robot screw locking according to claim 5 is characterized in that: The adjusting of the camera exposure parameters and the light source brightness according to the current ambient light intensity and the preset parameters includes: According to the preset target light intensity I tar , Initial setting brightness percentage of light source P set , Anti-oscillation proportional coefficient K p And the current ambient light intensity I act Calculate the light intensity output constraint P LED : Among them, the target light intensity I tar The unit is klux; P LED ∈[0%,100%]; According to the preset maximum exposure time T of the camera max , camera sensitivity constant C, target signal-to-noise ratio SNR tar And the current ambient light intensity I act , constraint G calculates the camera signal-to-noise ratio constraint T exp : The camera sensitivity constant C is expressed in μs·klux. 0.5 ; Camera maximum exposure time T max The unit is ms; the calculation formula of constraint condition G is as follows: Among them, G set is the preset gain; T min The minimum exposure time of the camera, in ms; The calculated light intensity output constraint P LED Send it to the light source module and constrain the calculated camera signal-to-noise ratio T exp , constraint condition G is sent to the camera module.

7. The visual intelligent calibration method for robot screw locking according to claim 6 is characterized in that: Also includes: When the constraint G exceeds a preset threshold, the target signal-to-noise ratio SNR is forced to be reduced. tar And recalculate the constraint G and the camera signal-to-noise ratio constraint T exp .

8. A visual intelligent calibration method for robot screw locking according to claim 6 or 7, characterized in that: Also includes: Obtain the signal-to-noise ratio constraint return value T from the light source module hardware register exp_read , and check that the error satisfies |T exp -T exp_read |≤0.5%, otherwise re-issue the parameters. If satisfied, continue to judge. If |SNR act -SNR tar |>2dB, the micro-compensation algorithm is triggered to iteratively adjust the target signal-to-noise ratio (SNR) of the current image ROI area with the set step size. tar Set the number of times.

Images