High-precision photoelectric imaging monitoring system and method based on image processing

Abstract

The invention discloses a high-precision photoelectric imaging monitoring system and method based on image processing, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a default shooting angle and light source configuration of a precise instrument of a corresponding model from a database, performing image acquisition on a precise instrument sample by using photoelectric imaging equipment, and importing the acquired image into image processing software for image preprocessing; identifying the contour of the precise instrument sample through edge detection, and detecting the shape feature of the precise instrument sample through transformation; segmenting a flaw area and a normal area of the precise instrument to be detected by using an image segmentation algorithm, and carrying out real-time flaw detection and classification by using a YOLO algorithm; light source configuration is optimized, the effect of each light source configuration is evaluated, and the optimal light source configuration is obtained; training a convolutional neural network, and determining an optimal shooting angle; under the optimal shooting angle and the optimal light source configuration, verification shooting is carried out on the to-be-tested precise instrument, the imaging quality is automatically evaluated, and feedback is provided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a high-precision photoelectric imaging monitoring system and method based on image processing. Background Art

[0002] In the field of precision instrument manufacturing and quality control, high-precision photoelectric imaging monitoring technology is widely used. Traditional photoelectric imaging monitoring methods usually rely on manual setting of shooting angles and light source configurations, which have the following disadvantages: Manual adjustment of the shooting angle and light source configuration is easily affected by the operator's experience and technical level, resulting in inconsistent imaging results. Existing technologies usually use simple image processing algorithms, which are difficult to efficiently and accurately identify tiny defects on precision instruments, resulting in missed detections or false detections. Light source configurations are usually not optimized according to the specific shape and characteristics of the instrument, which may lead to uneven lighting and thus affect imaging quality. Traditional methods lack automated image quality assessment methods and often rely on manual evaluation, which increases operational complexity and subjectivity. Summary of the invention

[0003] The purpose of the present invention is to provide a high-precision optoelectronic imaging monitoring system and method based on image processing to solve the problems raised in the prior art.

[0004] To achieve the above object, the present invention provides the following technical solutions: A high-precision photoelectric imaging monitoring method based on image processing, the steps of the method are as follows: S100, obtaining the model of the precision instrument to be tested, obtaining the default shooting angle and light source configuration of the precision instrument of the corresponding model from the database, using a photoelectric imaging device to collect images of the precision instrument sample, and importing the collected images into the image processing software for image preprocessing; S200, using Canny edge detection to identify the contour of precision instrument samples, and using Hough transform to detect the shape characteristics of precision instrument samples; using image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and using YOLO algorithm to perform real-time defect detection and classification through feature analysis; S300. According to the contour and shape characteristics of the precision instrument sample, the number and position of the light sources are adjusted by the particle swarm algorithm to optimize the light source configuration, and several light source configurations are designed for testing, and the effect of each light source configuration is evaluated to obtain the best light source configuration; the convolutional neural network is trained to avoid the defective area of ​​the precision instrument to be tested according to the best light source configuration, and the best shooting angle is determined according to historical data; S400, performs verification shooting of the precision instrument to be tested at the best shooting angle and the best light source configuration, and uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

[0005] According to step S100, the model of the precision instrument to be tested is obtained by scanning the QR code attached to the surface of the precision instrument to be tested, and the user is allowed to manually input the model; the database established in advance is connected to search for detailed information of the model of the precision instrument, and the default shooting angle and light source configuration of the precision instrument of the corresponding model are extracted; the settings of the optoelectronic imaging device are adjusted according to the default shooting angle and light source configuration; the angle, position and intensity of the light source are adjusted according to the default light source configuration; the optoelectronic imaging device is used to collect images of the precision instrument sample several times, the images are stored in TIFF format, and the parameter information of the image collection is recorded; The acquired images are uploaded from the optoelectronic imaging device to the image processing software for image preprocessing, denoising, removing the noise that occurs during the acquisition process, image enhancement, and adjusting the brightness and contrast of the image.

[0006] According to step S200, using Canny edge detection, the edge area of ​​potential precision instruments is identified by calculating the gradient strength and direction of each pixel in the image. The gradient represents the speed of image brightness change, and the edge appears where the gradient value is high. Non-maximum suppression is performed by comparing the gradient value of the current pixel with the values ​​of neighboring pixels in the gradient direction, retaining the local maximum value, and removing redundant edge pixels. In order to distinguish strong edges from weak edges, a double threshold processing method is adopted. Two thresholds are set. The gradient value of the strong edge is higher than the upper threshold and is clearly identified as the edge of the precision instrument; pixels with gradient values ​​lower than the lower threshold are excluded; pixels between the upper threshold and the lower threshold are marked as weak edges; weak edges are connected, and by traversing the image, the weak edge pixels adjacent to the strong edge are connected to form the outline of the precision instrument.

[0007] According to step S200, Hough transform is used to map points in the image space to the parameter space, and each edge point is converted into a curve in the parameter space, indicating that the point belongs to a number of shapes. Through mapping, the shape characteristics of the precision instrument sample are detected; in the parameter space, an accumulator is established to record the frequency of occurrence of each parameter combination. Whenever a curve passes through a certain point of the accumulator, the count value of the point increases, thereby accumulating information from edge points of several precision instrument samples; in the accumulator, the local maximum value is found, and the parameter combination with a high frequency is identified. The shape feature corresponding to the combination is the main geometric shape in the sample, and the shape characteristics of the precision instrument sample are determined.

[0008] According to step S200, the gray value of the image is divided into foreground and background by setting a threshold value, and the threshold value T is set so that: ; Among them, I is the gray value of the image, and img is the binary image after segmentation; Further, the adjacent pixel values ​​of similar areas are clustered into complete segmented areas, and the segmented areas can reflect the defective areas and normal areas. YOLO divides the input image into S*S grids, each grid is responsible for detecting whether there are defects in the image. If the center point of the defect falls in a certain grid, the grid will be responsible for detecting the defect, where S is a positive integer. Each grid predicts B bounding boxes, each of which contains four parameters: center point coordinates (a, b), width w, height h, and confidence score; where B is a positive integer. Through the neural network, YOLO predicts the probability distribution of each category and uses the Softmax function for normalization. YOLO uses a dual loss function, including: bounding box positioning loss and classification loss; the bounding box positioning loss formula is as follows: ; in, Indicates whether there is a defect obj in the B bounding boxes of mesh i, is the coefficient for adjusting coordinate errors, is the coordinate of the center point of the ground-truth bounding box of the i-th grid, is the coordinate of the center point of the i-th grid prediction bounding box, is the width of the ground-truth bounding box of the ith grid, is the width of the predicted bounding box of the ith grid, is the height of the true bounding box of the ith grid, is the height of the predicted bounding box of the i-th grid; The classification loss function formula is as follows: ; Where L is the total number of defect categories detected, is the probability of the true class, is the probability of the predicted class.

[0009] According to step S300, a boundary curve point set is extracted from the contour of the precision instrument sample. , use the curvature formula to calculate the surface curvature of precision instrument samples : ; in, is the first-order derivative of the curve, which represents the slope of the tangent line at a point on the curve. is the second derivative of the curve, which represents the acceleration of the curve at that point; Based on the shape characteristics of precision instrument samples, several light source configuration schemes are designed. The configuration optimization goal of the light source is to minimize the imaging distortion of reflections, shadows, and defective areas. The specific steps are as follows: Set the relative position parameters between the light source and the precision instrument sample, including the light source angle and the distance d are optimized by the following function: ,in, Indicates the reflectivity of the surface of the precision instrument sample, Indicates the area of ​​the shadow; The intensity of the light source LSI is adjusted according to the reflectivity of the surface of the precision instrument. The formula is as follows: ; in, is the initial intensity of the light source, r is the distance from the light source to the sample surface; For precision instrument samples with different shape characteristics, the number and distribution of light sources need to be uniform. The number and position of light sources are adjusted through the particle swarm algorithm, with the goal of finding a light source layout with uniform illumination. By testing different light source configurations, the advantages and disadvantages of each light source configuration scheme are evaluated based on contrast, clarity and defect detection accuracy to obtain the optimal light source configuration.

[0010] The number and position of light sources are used as optimization variables of the particle swarm algorithm. There are N light sources, and the three-dimensional coordinates of each light source are , where j = 1, 2, ..., N, represents the position of each light source; the goal is to find a light source layout that makes the light intensity distribution on the surface of the precision instrument uniform, and use the variance of the light intensity distribution as the objective function, denoted as , the calculation formula is as follows: ; Where M is the number of points sampled on the surface of the precision instrument, is the light intensity at the kth sampling point, is the light intensity at all sampling points, expressed as: , the goal is to minimize the variance of the light intensity distribution , the more uniform the illumination, the smaller the variance; The light intensity of each light source at a certain point on the surface of the precision instrument Calculated using the inverse square law: ; in, is the power of the jth light source, is the distance from light source j to the sampling point, calculated by Euclidean distance; The particle swarm algorithm searches for the optimal solution by simulating the position update of several particles in the search space. Each particle represents a layout of light sources, including the number and coordinates of light sources. N particles are randomly initialized in the search space, each particle contains the three-dimensional coordinates of N light sources, and the speed update formula of each particle is: ; in, is the velocity of the particle in the jth dimension, is the inertia weight, which controls the moving inertia of the particle. is the first acceleration coefficient, which controls the particle towards its personal optimal position The tendency of is the second acceleration factor, which controls the particles towards their optimal position in the group tendency of is the first random number, is the second random number, ranging from [0,1]; is the current position of the particle; The particle's position is updated based on its velocity as: ; Calculate the light intensity variance of each particle and update the individual optimal position and the optimal position of the group , repeat the velocity and position update steps until the convergence condition or the maximum number of iterations is reached.

[0011] According to step S300, historical data related to the shooting of precision instruments are collected, including different light source configurations, shooting angles, annotation information of defect areas, and defect conditions detected after each shooting; the collected historical data are pre-processed, and the historical data are divided into a training set and a test set, ensuring that the training set contains several light source configurations and shooting angles; The convolutional neural network model is trained using the training set. By inputting images containing defective areas and light source configurations, the model learns how to identify which areas are defects and feedbacks the best shooting angle based on historical data. During the training process, the regression method is used to predict the best shooting angle and optimize the model parameters. After the training is completed, the performance of the model is verified using the test set. By evaluating the performance of the model on the test set, the model's ability to recognize defective areas and the accuracy of predicting the best shooting angle are checked. The new light source configuration and image are input into the model, and the model will analyze the image, avoid defective areas, and output the best shooting angle.

[0012] According to step S400, the precision instrument is photographed using an optoelectronic imaging device according to the best shooting angle and the best light source configuration; after the shooting is completed, the acquired image is input into the image quality evaluation algorithm NIQE, and the image quality is automatically analyzed and evaluated without the need for the original image as a control; NIQE will analyze several quality indicators of the image to generate a quality score that reflects the clarity, naturalness and overall imaging quality of the image; based on the generated quality score, the final shooting effect is evaluated; if the score reaches the preset quality standard, it means that the imaging quality is qualified and the verification is passed; if the score is lower than the standard, feedback will be automatically generated to point out the problem; after receiving the feedback, the shooting angle and light source configuration will be adjusted according to the evaluation results.

[0013] NIQE first obtains information about image quality by analyzing the local features of the image. It mainly focuses on the following aspects: Calculate statistical features such as mean, variance, contrast, etc. of the image to understand the overall distribution of the image. Evaluate structural information in the image, such as edges, textures, and shapes, which have an important impact on visual quality. NIQE uses a set of statistical models trained on natural images to describe the characteristics of high-quality images. These models are based on different principles of visual perception and can capture the sensitive characteristics of the human eye to image quality. NIQE builds a benchmark model for quality assessment by learning the statistical characteristics of a large number of high-quality natural images.

[0014] After feature extraction and model building, NIQE compares the extracted features with the previously constructed high-quality image model and calculates the image quality score. The algorithm calculates the distance between the feature vector and the benchmark model to obtain a numerical score. The higher the score, the better the image quality, and vice versa.

[0015] Based on the calculated quality score, NIQE can provide direct feedback on image quality, helping to identify problems in the imaging process, such as noise, blur or distortion. This information can be used to adjust subsequent shooting settings or optimize the imaging process.

[0016] A high-precision photoelectric imaging monitoring system based on image processing, characterized by comprising: Image acquisition module: including: database query unit, image acquisition unit, image import unit and image preprocessing unit; wherein the database query unit obtains the model of the precision instrument to be tested, obtains the default shooting angle and light source configuration of the corresponding model of the precision instrument from the database, the image acquisition unit uses the photoelectric imaging device to collect images of the precision instrument samples, the image import unit imports the collected images into the image processing software, and the image preprocessing unit performs image preprocessing; Feature detection and defect classification module: including: edge detection unit, shape feature detection unit, image segmentation unit and defect detection unit; wherein, the edge detection unit uses Canny edge detection to identify the contour of the precision instrument sample, the shape feature detection unit uses Hough transform to detect the shape feature of the precision instrument sample, the image segmentation unit uses image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and the defect detection unit uses YOLO algorithm to perform real-time defect detection and classification through feature analysis; Light source optimization and shooting angle determination module: including: contour analysis unit, optimal configuration determination unit and shooting angle determination unit; wherein, the contour analysis unit adjusts the number and position of light sources according to the contour and shape characteristics of the precision instrument sample, and the optimal configuration determination unit adjusts the number and position of light sources through the particle swarm algorithm to optimize the light source configuration, designs several light source configurations for testing, evaluates the effect of each light source configuration, and obtains the optimal light source configuration; the shooting angle determination unit trains a convolutional neural network, avoids the defective area of ​​the precision instrument to be tested according to the optimal light source configuration, and determines the optimal shooting angle according to historical data; The final verification module includes: a verification shooting unit and an image quality assessment unit; the verification shooting unit performs verification shooting of the precision instrument to be tested at the optimal shooting angle and the optimal light source configuration, and the image quality assessment unit uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention combines Canny edge detection, Hough transform and YOLO algorithm to accurately identify the contour and defects of the instrument, thereby improving the accuracy and real-time performance of defect detection.

[0018] 2. The present invention designs and tests a variety of light source configurations, and uses a particle swarm algorithm to optimize the number and position of light sources to achieve uniform illumination and ensure imaging effects.

[0019] 3. The present invention introduces the NIQE algorithm for non-reference image quality assessment, which can automatically provide imaging quality feedback, provide a basis for subsequent improvements, and reduce manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the steps of a high-precision photoelectric imaging monitoring method based on image processing of the present invention; Figure 2 It is a system structure diagram of a high-precision photoelectric imaging monitoring system based on image processing of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution. According to one embodiment of the present invention, Figure 1 A high-precision photoelectric imaging monitoring method based on image processing is shown in a schematic diagram of the steps of the method. A high-precision photoelectric imaging monitoring method based on image processing, the steps of the method are as follows: S100, obtaining the model of the precision instrument to be tested, obtaining the default shooting angle and light source configuration of the precision instrument of the corresponding model from the database, using a photoelectric imaging device to collect images of the precision instrument sample, and importing the collected images into the image processing software for image preprocessing; S200, using Canny edge detection to identify the contour of precision instrument samples, and using Hough transform to detect the shape characteristics of precision instrument samples; using image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and using YOLO algorithm to perform real-time defect detection and classification through feature analysis; S300. According to the contour and shape characteristics of the precision instrument sample, the number and position of the light sources are adjusted by the particle swarm algorithm to optimize the light source configuration, and several light source configurations are designed for testing, and the effect of each light source configuration is evaluated to obtain the best light source configuration; the convolutional neural network is trained to avoid the defective area of ​​the precision instrument to be tested according to the best light source configuration, and the best shooting angle is determined according to historical data; S400, performs verification shooting of the precision instrument to be tested at the best shooting angle and the best light source configuration, and uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

[0023] According to step S100, the model of the precision instrument to be tested is obtained by scanning the QR code attached to the surface of the precision instrument to be tested, and the user is allowed to manually input the model; the database established in advance is connected to search for detailed information of the model of the precision instrument, and the default shooting angle and light source configuration of the precision instrument of the corresponding model are extracted; the settings of the optoelectronic imaging device are adjusted according to the default shooting angle and light source configuration; the angle, position and intensity of the light source are adjusted according to the default light source configuration; the optoelectronic imaging device is used to collect images of the precision instrument sample several times, the images are stored in TIFF format, and the parameter information of the image collection is recorded; The acquired images are uploaded from the optoelectronic imaging device to the image processing software for image preprocessing, denoising, removing the noise that occurs during the acquisition process, image enhancement, and adjusting the brightness and contrast of the image.

[0024] According to step S200, using Canny edge detection, the edge area of ​​potential precision instruments is identified by calculating the gradient strength and direction of each pixel in the image. The gradient represents the speed of image brightness change, and the edge appears where the gradient value is high. Non-maximum suppression is performed by comparing the gradient value of the current pixel with the values ​​of neighboring pixels in the gradient direction, retaining the local maximum value, and removing redundant edge pixels. In order to distinguish strong edges from weak edges, a double threshold processing method is adopted. Two thresholds are set. The gradient value of the strong edge is higher than the upper threshold and is clearly identified as the edge of the precision instrument; pixels with gradient values ​​lower than the lower threshold are excluded; pixels between the upper threshold and the lower threshold are marked as weak edges; weak edges are connected, and by traversing the image, the weak edge pixels adjacent to the strong edge are connected to form the outline of the precision instrument.

[0025] According to step S200, Hough transform is used to map points in the image space to the parameter space, and each edge point is converted into a curve in the parameter space, indicating that the point belongs to a number of shapes. Through mapping, the shape characteristics of the precision instrument sample are detected; in the parameter space, an accumulator is established to record the frequency of occurrence of each parameter combination. Whenever a curve passes through a certain point of the accumulator, the count value of the point increases, thereby accumulating information from edge points of several precision instrument samples; in the accumulator, the local maximum value is found, and the parameter combination with a high frequency is identified. The shape feature corresponding to the combination is the main geometric shape in the sample, and the shape characteristics of the precision instrument sample are determined.

[0026] According to step S200, in this embodiment, 100 images of precision instruments are selected, which contain different types of defects, such as scratches, stains, etc. By setting a threshold, the gray value of the image is divided into foreground and background, and the threshold T is set to 128, so that: ; Among them, I is the gray value of the image, and img is the binary image after segmentation; Further, the adjacent pixel values ​​of similar areas are clustered into complete segmented areas, and the segmented areas can reflect the defective areas and normal areas. YOLO divides the input image into S*S grids, each grid is responsible for detecting whether there is a defect in the image. If the center point of the defect falls in a certain grid, the grid will be responsible for detecting the defect, where S is 7. Each grid predicts B bounding boxes, each of which contains four parameters: center point coordinates (a, b), width w, height h, and confidence score; where B is 2. Through the neural network, YOLO predicts the probability distribution of each category and uses the Softmax function for normalization. YOLO uses a dual loss function, including: bounding box positioning loss and classification loss; the bounding box positioning loss formula is as follows: ; in, Indicates whether there is a defect obj in the B bounding boxes of mesh i, is the coefficient for adjusting coordinate errors, is the coordinate of the center point of the ground-truth bounding box of the i-th grid, is the coordinate of the center point of the i-th grid prediction bounding box, is the width of the ground-truth bounding box of the ith grid, is the width of the predicted bounding box of the ith grid, is the height of the true bounding box of the ith grid, is the height of the predicted bounding box of the i-th grid; The classification loss function formula is as follows:

[0027] Where L is the total number of defect categories detected, which is 3, including scratches, stains, and others. is the probability of the true class, is the probability of the predicted class.

[0028] The model has a detection accuracy of 92% for 30 images in the test set. The average error of bounding box positioning is 5.3 pixels, indicating that the model is relatively accurate in locating defects. For the detected defects: scratch detection accuracy: 90%, stain detection accuracy: 95%, other defect detection accuracy: 85%.

[0029] According to step S300, in this embodiment, 10 precision instrument samples with different shape features are selected, including cylindrical, square and complex shapes. The initial light source intensity is 1000 Lux, the distance d includes different settings from 20 cm to 100 cm, and the light source angle includes different configurations such as 0°, 30°, 60° and 90°; the boundary curve point set is extracted from the contour of the precision instrument sample , we have a set of boundary curve points of a precision instrument sample, take the coordinates of the following 5 points: point A (1,2), point B (2,3), point C (3,5), point D (4,4), point E (5,2). Use the curvature formula to calculate the surface curvature of the precision instrument sample : ; in, is the first-order derivative of the curve, which represents the slope of the tangent line at a point on the curve. is the second derivative of the curve, which represents the acceleration of the curve at that point; Perform the first-order derivative calculation at point B: , at point C: , at point D: ; Calculate the second-order derivative at point C: , at point D: ; Calculate the curvature at point B: , at point C: , at point D: .

[0030] Based on the shape characteristics of precision instrument samples, several light source configuration schemes are designed. The configuration optimization goal of the light source is to minimize the imaging distortion of reflections, shadows, and defective areas. The specific steps are as follows: Set the relative position parameters between the light source and the precision instrument sample, including the light source angle and the distance d are optimized by the following function: ,in, Indicates the reflectivity of the surface of the precision instrument sample, Indicates the area of ​​the shadow; The intensity of the light source LSI is adjusted according to the reflectivity of the surface of the precision instrument. The formula is as follows: ; in, is the initial intensity of the light source, r is the distance from the light source to the sample surface; For precision instrument samples with different shape characteristics, the number and distribution of light sources need to be uniform. The number and position of light sources are adjusted through the particle swarm algorithm, with the goal of finding a light source layout with uniform illumination. By testing different light source configurations, the advantages and disadvantages of each light source configuration scheme are evaluated based on contrast, clarity and defect detection accuracy to obtain the optimal light source configuration.

[0031] During the algorithm operation, the optimization results are as follows: the number of light sources is 4, and the distribution positions include: light source 1 (0, 1), light source 2 (1, 0), light source 3 (1, 2), and light source 4 (2, 1).

[0032] The number and position of light sources are used as optimization variables of the particle swarm algorithm. There are N light sources, and the three-dimensional coordinates of each light source are , where j = 1, 2, ..., N, represents the position of each light source; the goal is to find a light source layout that makes the light intensity distribution on the surface of the precision instrument uniform, and use the variance of the light intensity distribution as the objective function, denoted as , the calculation formula is as follows: ; Where M is the number of points sampled on the surface of the precision instrument, is the light intensity at the kth sampling point, is the light intensity at all sampling points, expressed as: , the goal is to minimize the variance of the light intensity distribution , the more uniform the illumination, the smaller the variance; The light intensity of each light source at a certain point on the surface of the precision instrument Calculated using the inverse square law: ; in, is the power of the jth light source, is the distance from light source j to the sampling point, calculated by Euclidean distance; The particle swarm algorithm searches for the optimal solution by simulating the position update of several particles in the search space. Each particle represents a layout of light sources, including the number and coordinates of light sources. N particles are randomly initialized in the search space, each particle contains the three-dimensional coordinates of N light sources, and the speed update formula of each particle is: ; in, is the velocity of the particle in the jth dimension, is the inertia weight, which controls the moving inertia of the particle. is the first acceleration coefficient, which controls the particle towards its personal optimal position The tendency of is the second acceleration factor, which controls the particles towards their optimal position in the group tendency of is the first random number, is the second random number, ranging from [0,1]; is the current position of the particle; The particle's position is updated based on its velocity as: ; Calculate the light intensity variance of each particle and update the individual optimal position and the optimal position of the group , repeat the velocity and position update steps until the convergence condition or the maximum number of iterations is reached.

[0033] According to step S300, historical data related to the shooting of precision instruments are collected, including different light source configurations, shooting angles, annotation information of defect areas, and defect conditions detected after each shooting; the collected historical data are pre-processed, and the historical data are divided into a training set and a test set, ensuring that the training set contains several light source configurations and shooting angles; The convolutional neural network model is trained using the training set. By inputting images containing defective areas and light source configurations, the model learns how to identify which areas are defects and feedbacks the best shooting angle based on historical data. During the training process, the regression method is used to predict the best shooting angle and optimize the model parameters. After the training is completed, the performance of the model is verified using the test set. By evaluating the performance of the model on the test set, the model's ability to recognize defective areas and the accuracy of predicting the best shooting angle are checked. The new light source configuration and image are input into the model, and the model will analyze the image, avoid defective areas, and output the best shooting angle.

[0034] According to step S400, the precision instrument is photographed using an optoelectronic imaging device according to the best shooting angle and the best light source configuration; after the shooting is completed, the acquired image is input into the image quality evaluation algorithm NIQE, and the image quality is automatically analyzed and evaluated without the need for the original image as a control; NIQE will analyze several quality indicators of the image to generate a quality score that reflects the clarity, naturalness and overall imaging quality of the image; based on the generated quality score, the final shooting effect is evaluated; if the score reaches the preset quality standard, it means that the imaging quality is qualified and the verification is passed; if the score is lower than the standard, feedback will be automatically generated to point out the problem; after receiving the feedback, the shooting angle and light source configuration will be adjusted according to the evaluation results.

[0035] According to another embodiment of the present invention, Figure 2 As shown in the system structure diagram of a high-precision optoelectronic imaging monitoring system based on image processing, a high-precision optoelectronic imaging monitoring system based on image processing is characterized in that it includes: Image acquisition module: including: database query unit, image acquisition unit, image import unit and image preprocessing unit; wherein the database query unit obtains the model of the precision instrument to be tested, obtains the default shooting angle and light source configuration of the corresponding model of the precision instrument from the database, the image acquisition unit uses the photoelectric imaging device to collect images of the precision instrument samples, the image import unit imports the collected images into the image processing software, and the image preprocessing unit performs image preprocessing; Feature detection and defect classification module: including: edge detection unit, shape feature detection unit, image segmentation unit and defect detection unit; wherein, the edge detection unit uses Canny edge detection to identify the contour of the precision instrument sample, the shape feature detection unit uses Hough transform to detect the shape feature of the precision instrument sample, the image segmentation unit uses image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and the defect detection unit uses YOLO algorithm to perform real-time defect detection and classification through feature analysis; Light source optimization and shooting angle determination module: including: contour analysis unit, optimal configuration determination unit and shooting angle determination unit; wherein, the contour analysis unit adjusts the number and position of light sources according to the contour and shape characteristics of the precision instrument sample, and the optimal configuration determination unit adjusts the number and position of light sources through the particle swarm algorithm to optimize the light source configuration, designs several light source configurations for testing, evaluates the effect of each light source configuration, and obtains the optimal light source configuration; the shooting angle determination unit trains a convolutional neural network, avoids the defective area of ​​the precision instrument to be tested according to the optimal light source configuration, and determines the optimal shooting angle according to historical data; In this embodiment, the sample contour data: contour point set: [(0, 0), (1, 2), (2, 2), (3, 1), (2, 0), (1, 0)], surface feature: curvature range [0.1, 0.5]. The light source configuration parameters are: initial light source intensity 1000 Lux, surface reflectivity 0.85, light source distance 1.5 meters.

[0036] By analyzing the contour data, the best light source layout is selected, and light sources with different angles (30°, 45°, 60°) and positions are tried. Configuration 1 (2 light sources): contrast 0.65, clarity 0.70, defect detection accuracy 78%; Configuration 2 (4 light sources): contrast 0.85, clarity 0.90, defect detection accuracy 92%. Six light source configurations were designed and evaluated, and the best configuration was configuration 2 (4 light sources). Experimental data summary: Configuration 1 (2 light sources): contrast: 0.65, clarity: 0.70, defect detection accuracy: 78%; Configuration 2 (4 light sources): contrast: 0.85, clarity: 0.90, defect detection accuracy: 92%. After training, the best shooting angle predicted by CNN is 45°. The prediction accuracy is 95%, which is based on historical data.

[0037] The final verification module includes: a verification shooting unit and an image quality assessment unit; the verification shooting unit performs verification shooting of the precision instrument to be tested at the optimal shooting angle and the optimal light source configuration, and the image quality assessment unit uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

[0038] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A high-precision photoelectric imaging monitoring method based on image processing, characterized in that: The steps of the method are as follows: S100, obtaining the model of the precision instrument to be tested, obtaining the default shooting angle and light source configuration of the precision instrument of the corresponding model from the database, using a photoelectric imaging device to collect images of the precision instrument sample, and importing the collected images into the image processing software for image preprocessing; S200, using Canny edge detection to identify the contour of precision instrument samples, and using Hough transform to detect the shape characteristics of precision instrument samples; using image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and using YOLO algorithm to perform real-time defect detection and classification through feature analysis; S300. According to the contour and shape characteristics of the precision instrument sample, the number and position of the light sources are adjusted by the particle swarm algorithm to optimize the light source configuration, and several light source configurations are designed for testing, and the effect of each light source configuration is evaluated to obtain the best light source configuration; the convolutional neural network is trained to avoid the defective area of ​​the precision instrument to be tested according to the best light source configuration, and the best shooting angle is determined according to historical data; S400, performs verification shooting of the precision instrument to be tested at the best shooting angle and the best light source configuration, and uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

2. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 1 is characterized in that: According to step S100, the model of the precision instrument to be tested is obtained by scanning the QR code attached to the surface of the precision instrument to be tested, and the user is allowed to manually input the model; the database established in advance is connected to search for detailed information of the model of the precision instrument, and the default shooting angle and light source configuration of the precision instrument of the corresponding model are extracted; the settings of the optoelectronic imaging device are adjusted according to the default shooting angle and light source configuration; the angle, position and intensity of the light source are adjusted according to the default light source configuration; the optoelectronic imaging device is used to collect images of the precision instrument sample several times, the images are stored in TIFF format, and the parameter information of the image collection is recorded; The acquired images are uploaded from the optoelectronic imaging device to the image processing software for image preprocessing, denoising, removing the noise that occurs during the acquisition process, image enhancement, and adjusting the brightness and contrast of the image.

3. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 1 is characterized in that: According to step S200, using Canny edge detection, the edge area of ​​potential precision instruments is identified by calculating the gradient strength and direction of each pixel in the image. The gradient represents the speed of image brightness change, and the edge appears where the gradient value is high. Non-maximum suppression is performed by comparing the gradient value of the current pixel with the values ​​of neighboring pixels in the gradient direction, retaining the local maximum value, and removing redundant edge pixels. In order to distinguish strong edges from weak edges, a double threshold processing method is adopted. Two thresholds are set. The gradient value of the strong edge is higher than the upper threshold and is clearly identified as the edge of the precision instrument; pixels with gradient values ​​lower than the lower threshold are excluded; pixels between the upper threshold and the lower threshold are marked as weak edges; weak edges are connected, and by traversing the image, the weak edge pixels adjacent to the strong edge are connected to form the outline of the precision instrument.

4. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 3 is characterized in that: According to step S200, Hough transform is used to map points in the image space to the parameter space, and each edge point is converted into a curve in the parameter space, indicating the condition that the point belongs to several shapes. Through mapping, the shape characteristics of the precision instrument sample are detected; In the parameter space, an accumulator is established to record the frequency of occurrence of each parameter combination. Whenever a curve passes through a point in the accumulator, the count value of the point increases, thereby accumulating information from edge points of several precision instrument samples. In the accumulator, local maxima are found and parameter combinations with high frequencies are identified. The shape features corresponding to the combinations are the main geometric shapes in the samples, and the shape features of the precision instrument samples are determined.

5. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 4 is characterized in that: According to step S200, the gray value of the image is divided into foreground and background by setting a threshold value, and the threshold value T is set so that: ; Among them, I is the gray value of the image, and img is the binary image after segmentation; Further, the adjacent pixel values ​​of similar areas are clustered into complete segmented areas, and the segmented areas can reflect the defective areas and normal areas. YOLO divides the input image into S*S grids, each grid is responsible for detecting whether there are defects in the image. If the center point of the defect falls in a certain grid, the grid will be responsible for detecting the defect, where S is a positive integer. Each grid predicts B bounding boxes, each of which contains four parameters: center point coordinates (a, b), width w, height h, and confidence score; where B is a positive integer. Through the neural network, YOLO predicts the probability distribution of each category and uses the Softmax function for normalization. YOLO uses a dual loss function, including: bounding box positioning loss and classification loss; the bounding box positioning loss formula is as follows: ； in, Indicates whether there is a defect obj in the B bounding boxes of mesh i, is the coefficient for adjusting coordinate errors, is the coordinate of the center point of the ground-truth bounding box of the i-th grid, is the coordinate of the center point of the i-th grid prediction bounding box, is the width of the ground-truth bounding box of the ith grid, is the width of the predicted bounding box of the ith grid, is the height of the true bounding box of the ith grid, is the height of the predicted bounding box of the i-th grid; The classification loss function formula is as follows: ； Where L is the total number of defect categories detected, is the probability of the true class, is the probability of the predicted class.

6. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 1 is characterized in that: According to step S300, a boundary curve point set is extracted from the contour of the precision instrument sample. , use the curvature formula to calculate the surface curvature of precision instrument samples : ； in, is the first-order derivative of the curve, which represents the slope of the tangent line at a point on the curve. is the second derivative of the curve, which represents the acceleration of the curve at that point; Based on the shape characteristics of precision instrument samples, several light source configuration schemes are designed. The configuration optimization goal of the light source is to minimize the imaging distortion of reflections, shadows, and defective areas. The specific steps are as follows: Set the relative position parameters between the light source and the precision instrument sample, including the light source angle and the distance d are optimized by the following function: ,in, Indicates the reflectivity of the surface of the precision instrument sample, Indicates the area of ​​the shadow; The intensity of the light source LSI is adjusted according to the reflectivity of the surface of the precision instrument. The formula is as follows: ； in, is the initial intensity of the light source, r is the distance from the light source to the sample surface; For precision instrument samples with different shape characteristics, the number and distribution of light sources need to be uniform. The number and position of light sources are adjusted through the particle swarm algorithm, with the goal of finding a light source layout with uniform illumination. By testing different light source configurations, the advantages and disadvantages of each light source configuration scheme are evaluated based on contrast, clarity and defect detection accuracy to obtain the optimal light source configuration.

7. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 6 is characterized in that: The number and position of light sources are used as optimization variables of the particle swarm algorithm. There are N light sources, and the three-dimensional coordinates of each light source are , where j = 1, 2, ..., N, represents the position of each light source; the goal is to find a light source layout that makes the light intensity distribution on the surface of the precision instrument uniform, and use the variance of the light intensity distribution as the objective function, denoted as , the calculation formula is as follows: ； Where M is the number of points sampled on the surface of the precision instrument, is the light intensity at the kth sampling point, is the light intensity at all sampling points, expressed as: , the goal is to minimize the variance of the light intensity distribution , the more uniform the illumination, the smaller the variance; The light intensity of each light source at a certain point on the surface of the precision instrument Calculated using the inverse square law: ； in, is the power of the jth light source, is the distance from light source j to the sampling point, calculated by Euclidean distance; The particle swarm algorithm searches for the optimal solution by simulating the position update of several particles in the search space. Each particle represents a layout of light sources, including the number and coordinates of light sources. N particles are randomly initialized in the search space, each particle contains the three-dimensional coordinates of N light sources, and the speed update formula of each particle is: ； in, is the velocity of the particle in the jth dimension, is the inertia weight, which controls the moving inertia of the particle. is the first acceleration coefficient, which controls the particle towards its personal optimal position The tendency of is the second acceleration factor, which controls the particles towards their group's optimal position tendency of is the first random number, is the second random number, ranging from [0,1]; is the current position of the particle; The particle's position is updated based on its velocity as: ； Calculate the light intensity variance of each particle and update the individual optimal position and the optimal position of the group , repeat the velocity and position update steps until the convergence condition or the maximum number of iterations is reached.

8. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 7 is characterized in that: According to step S300, historical data related to the shooting of precision instruments are collected, including different light source configurations, shooting angles, annotation information of defect areas, and defect conditions detected after each shooting; the collected historical data are pre-processed, and the historical data are divided into a training set and a test set, ensuring that the training set contains several light source configurations and shooting angles; The convolutional neural network model is trained using the training set. By inputting images containing defective areas and light source configurations, the model learns how to identify which areas are defects and feedbacks the best shooting angle based on historical data. During the training process, the regression method is used to predict the best shooting angle and optimize the model parameters. After the training is completed, the performance of the model is verified using the test set. By evaluating the performance of the model on the test set, the model's ability to recognize defective areas and the accuracy of predicting the best shooting angle are checked. The new light source configuration and image are input into the model, and the model will analyze the image, avoid defective areas, and output the best shooting angle.

9. The high-precision optoelectronic imaging monitoring method based on image processing according to claim 1, characterized in that: According to step S400, the precision instrument is photographed using an optoelectronic imaging device according to the best shooting angle and the best light source configuration; after the shooting is completed, the acquired image is input into the image quality evaluation algorithm NIQE, and the image quality is automatically analyzed and evaluated without the need for the original image as a control; NIQE will analyze several quality indicators of the image to generate a quality score that reflects the clarity, naturalness and overall imaging quality of the image; based on the generated quality score, the final shooting effect is evaluated; if the score reaches the preset quality standard, it means that the imaging quality is qualified and the verification is passed; if the score is lower than the standard, feedback will be automatically generated to point out the problem; after receiving the feedback, the shooting angle and light source configuration will be adjusted according to the evaluation results.

10. A high-precision photoelectric imaging monitoring system based on image processing, using a high-precision photoelectric imaging monitoring method based on image processing according to any one of claims 1 to 9, characterized in that: include: Image acquisition module: including: database query unit, image acquisition unit, image import unit and image preprocessing unit; wherein the database query unit obtains the model of the precision instrument to be tested, obtains the default shooting angle and light source configuration of the corresponding model of the precision instrument from the database, the image acquisition unit uses the photoelectric imaging device to collect images of the precision instrument samples, the image import unit imports the collected images into the image processing software, and the image preprocessing unit performs image preprocessing; Feature detection and defect classification module: including: edge detection unit, shape feature detection unit, image segmentation unit and defect detection unit; wherein, the edge detection unit uses Canny edge detection to identify the contour of the precision instrument sample, the shape feature detection unit uses Hough transform to detect the shape feature of the precision instrument sample, the image segmentation unit uses image segmentation algorithm to segment the defect area and normal area of ​​the precision instrument to be tested, and the defect detection unit uses YOLO algorithm to perform real-time defect detection and classification through feature analysis; Light source optimization and shooting angle determination module: including: contour analysis unit, optimal configuration determination unit and shooting angle determination unit; wherein, the contour analysis unit adjusts the number and position of light sources according to the contour and shape characteristics of the precision instrument sample, and the optimal configuration determination unit adjusts the number and position of light sources through the particle swarm algorithm to optimize the light source configuration, designs several light source configurations for testing, evaluates the effect of each light source configuration, and obtains the optimal light source configuration; the shooting angle determination unit trains a convolutional neural network, avoids the defective area of ​​the precision instrument to be tested according to the optimal light source configuration, and determines the optimal shooting angle according to historical data; The final verification module includes: a verification shooting unit and an image quality assessment unit; the verification shooting unit performs verification shooting of the precision instrument to be tested at the optimal shooting angle and the optimal light source configuration, and the image quality assessment unit uses the image quality assessment algorithm NIQE to automatically evaluate the imaging quality and provide feedback for the final solution.

Images