Intelligent vision-based detection method and system, electronic device and storage medium

Abstract

The present application belongs to the technical field of intelligent vision detection, and discloses a detection method and system based on intelligent vision, an electronic device and a storage medium; comprising: collecting an image of an art derivative to be detected, the image of the art derivative to be detected being collected according to a collection interval t; inputting the image of the art derivative to be detected into a trained scratch detection model, outputting an artistic annotation result, and judging whether a scratch instruction with an AIGC artistic label is generated; according to the scratch instruction, the corresponding scratch degree is calculated; inputting the image of the art derivative to be detected into a trained fragmentation analysis model, outputting a fragmentation recognition result and an artistic value evaluation score, and judging whether a fragmentation instruction is generated; the present application improves product quality and production efficiency, and simultaneously controls artistic value, thereby meeting the comprehensive needs in the art derivative detection scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent visual detection, and more particularly, to a detection method and system based on intelligent vision, an electronic device and a storage medium. BACKGROUND

[0002] In the production line of art derivative products to be detected in the field of visual arts, with the large-scale production of ceramic bottles, glass sculptures, hand-drawn creative boxes and other products, the demand for accurate detection of product surface scratches, cracks and other damages is increasingly urgent. Such products need to ensure physical integrity while taking into account their unique visual art attributes to avoid losing artistic value due to improper detection. Existing intelligent detection technologies are mostly based on the detection logic of ordinary industrial products. The technical principle is as follows: an image acquisition device is used to obtain an image of the object to be detected, and after preprocessing such as background removal and grayscale, basic visual features such as shape and texture are extracted. A pre-set defect judgment threshold or a basic machine learning model is used to analyze the features, and a "qualified" or "unqualified" judgment result is output. The data samples used in the model training process are mainly damage images of ordinary industrial products such as ordinary glass bottles and metal parts, and are not adapted to the characteristics of art derivative products.

[0003] However, when such technology is applied to the detection of art derivative products to be detected, there are significant adaptation defects: on the one hand, art derivative products to be detected usually have specific visual art features such as blue and white hand-drawing and star and ray carving. The damage patterns such as fine cracks along the carving texture are essentially different from the regular damage of ordinary industrial products. The feature extraction algorithm of the existing technology cannot effectively distinguish between art texture and damage features, and is prone to misjudgment of art texture as damage or omission of slight damage along the art texture. On the other hand, the existing technology only uses the degree of physical damage as the basis for judgment and does not consider the impact of damage on the artistic value of art derivative products. It may judge the damage patterns such as ice cracks with unique artistic style and in line with ceramic aesthetic standards as unqualified products. In addition, the detection result output by the technology is only a simple pass or fail judgment, and does not include information such as artistic labeling of damage location, analysis of the impact of damage on artistic value, and cannot meet the needs of artistic review, damage tracing and detection result display in the production process of art derivative products to be detected.

[0004] The aforementioned problems directly result in a high rate of false positives and false negatives when existing technologies are applied to the production line of art derivatives to be inspected. This leads to the wrong rejection of qualified products or the entry of unqualified products into the market, which reduces production efficiency and affects product quality. Furthermore, it may lead to the misjudgment of products with artistic features due to the neglect of artistic value, resulting in a double loss of artistic value and economic benefits. Ultimately, the existing intelligent inspection technology is seriously inadequate in the inspection scenario of art derivatives to be inspected in the field of visual arts. It is difficult to support the comprehensive requirements of the production line for inspection accuracy, control of artistic value, and visual presentation of inspection results, thus restricting the quality control level of large-scale production of art derivatives to be inspected in the field of visual arts.

[0005] In view of this, the present invention proposes a detection method, system, electronic device and storage medium based on intelligent vision to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a detection method based on intelligent vision, comprising:

[0007] Images of the art derivatives to be detected are collected, and these images are collected at intervals t.

[0008] Input the image of the art derivative to be detected into the trained scratch detection model, output the artistic annotation result, and determine whether to generate a scratch instruction with an AIGC art tag;

[0009] Calculate the corresponding scratch severity based on the scratch instruction;

[0010] Input the image of the art derivative to be detected into the trained fragmentation analysis model, output the fragmentation recognition result and art value assessment score, and determine whether to generate a fragmentation command.

[0011] Calculate the fracture area of ​​the corresponding art derivative to be tested based on the fracture command;

[0012] The damage coefficient is obtained by combining the art value assessment score. The tolerance coefficient is calculated based on the damage coefficient and the art coefficient output by AIGC. The corresponding tolerance coefficient is then calculated based on the damage coefficient. Based on the tolerance coefficient, the RCS robot is controlled to remove the corresponding art derivative to be tested from the production line.

[0013] Furthermore, the image of the art derivative to be detected is the image of the art derivative to be detected in the production line. One image of the art derivative to be detected includes n art derivatives to be detected, where n is an integer greater than 1.

[0014] Methods for obtaining the sampling interval t include:

[0015] The moving speed is the running speed of the production line, and is obtained by calculating the pulse number, the pulse time and the resolution, wherein the pulse number is the number of pulse signals output by the encoder within the pulse time, the pulse time is a preset pulse signal collection time, and the resolution is a displacement corresponding to each pulse signal; and the expression of the moving speed is: , wherein is the moving speed, is the pulse number, is the resolution, is the pulse time.

[0016] The length data includes the diameter of the art derivative to be detected and the distance between adjacent art derivatives to be detected, wherein the diameter of the art derivative to be detected is the diameter of the bottom of the art derivative to be detected, and the distance between adjacent art derivatives to be detected is the distance between adjacent art derivatives to be detected; the distance between adjacent art derivatives to be detected is obtained by calculating the moving speed, the collection number, the collection time and the diameter of the art derivative to be detected, the collection number is a preset collection number of art derivatives to be detected, the collection time is the time experienced by the photoelectric sensor when the number of art derivatives to be detected collected reaches the collection number; and the expression of the distance between adjacent art derivatives to be detected is: , wherein is the distance between adjacent art derivatives to be detected, is the collection time, is the collection number, is the diameter of the art derivative to be detected.

[0017] The collection interval t is calculated according to the moving speed and the length data, and the expression of the collection interval t is: .

[0018] Further, the method for judging whether to generate a scratch instruction comprises:

[0019] Step a. inputting the image of the art derivative to be detected into the trained scratch detection model, and performing rectangular frame labeling on the n art derivatives to be detected in the image of the art derivative to be detected, that is, one-to-one correspondence between the n rectangular frames and the n art derivatives to be detected;

[0020] Step b. labeling the image of the art derivative to be detected containing the rectangular frame labeling as a labeled image, cutting the labeled image into n images according to the n rectangular frames, and marking the n images as analysis images;

[0021] Step c. Gray processing is performed on the n analysis images respectively, and it is judged whether there is a scratch in the n analysis images; if there is a scratch in the analysis image, a scratch instruction is generated, and the analysis image corresponding to the scratch instruction is marked as a scratch image; if there is no scratch in the analysis image, no scratch instruction is generated.

[0022] Further, the training method of the scratch detection model in step a comprises:

[0023] The image to be detected is marked as a first training image; supplementary training images with multiple damage types and scratches consistent with the artistic style of the image to be detected are generated by AIGC; the damage positions in the supplementary training images are labeled with rectangular frames; the first training images and the supplementary training images are divided into a training set and a test set, the training set is used to train the scratch detection model, and the test set is used to test the scratch detection model; a preset coordinate error threshold and an artistic damage recognition accuracy threshold, when the mean of the prediction coordinate error of all first training images in the test set is less than the coordinate error threshold, and the artistic damage recognition accuracy is higher than the artistic damage recognition accuracy threshold, the scratch detection model training is completed, and the scratch detection model is output; the scratch detection model is a convolutional neural network model.

[0024] Further, the method of cutting the labeled image into n images according to the n rectangular frames in step b comprises:

[0025] Read the image to be detected containing rectangular frame labels and rectangular frame coordinate files; extract the coordinates of the upper left corner and the lower right corner of each rectangular frame, and cut n analysis images from the labeled image according to the coordinates corresponding to the n rectangular frames.

[0026] Further, the method of judging whether there is a scratch in the n analysis images in step c comprises:

[0027] Collect the gray values of Y pixel points corresponding to each analysis image , Y is the number of all pixel points in an analysis image; is the gray value of the jth pixel point in the ith analysis image ;

[0028] A preset gray value threshold , ; if , the corresponding pixel point is marked as a normal point, if , the corresponding pixel point is marked as a scratch point; if there is a scratch point in an analysis image, it is judged that there is a scratch in the corresponding analysis image; otherwise, it is judged that there is no scratch in the corresponding analysis image.

[0029] Further, the scratch degree is calculated and obtained according to the scratch area and the number of scratches.

[0030] The method for obtaining the scratch area comprises:

[0031] The number of scratch points is planned, and the scratch area corresponding to each scratch image is calculated; the expression of the scratch area is: , wherein is the scratch area of the vth scratch image, is the number of scratch points of the vth scratch image, is the area of one pixel point, is a proportional coefficient, , and W is the number of scratch images.

[0032] The area of one pixel point is obtained from the resolution of the to-be-detected artistic derivative image, and the resolution of the to-be-detected artistic derivative image is obtained from the attribute of the to-be-detected artistic derivative image; the proportional coefficient is obtained by dividing the actual area of the to-be-detected artistic derivative by the area of the to-be-detected artistic derivative image , the actual area of the to-be-detected artistic derivative is the area corresponding to one unfragmented to-be-detected artistic derivative, and the area of the to-be-detected artistic derivative image is the image area corresponding to one unfragmented to-be-detected artistic derivative in the to-be-detected artistic derivative image;

[0033] The method for obtaining the number of scratches comprises:

[0034] W arrays consistent with the size of the scratch image are created as heat arrays, wherein the heat arrays correspond to the scratch images one by one, and a zero matrix is constructed; the size of the scratch image is the width and height of the scratch image.

[0035] A rectangular coordinate system is established with any vertex in each scratch image as the origin, and the origin of each scratch image is the same vertex; the coordinates of each scratch point in each scratch image are obtained according to the rectangular coordinate system; the elements at the corresponding positions in the heat array are assigned to according to the coordinates of each scratch point in each scratch image and the index of the corresponding heat array.

[0036] Gaussian blur is performed on the W heat arrays, the Gaussian kernel size is specified as , and the standard deviation is ; the W arrays output after Gaussian blur are respectively overlaid on the original heat arrays, and are marked as blurred arrays.

[0037] A line segment detection algorithm is used to detect line segments in the W blurred arrays, and the number of detected line segments in each blurred array is the number of scratches of the corresponding scratch image.

[0038] The degree of scratching is obtained by weighted summation of the scratch area and the number of scratches of the scratch image.

[0039] Further, the method of judging whether to generate the fragmentation instruction comprises:

[0040] Using the trained fragmentation analysis model, n analysis images in the to-be-detected artistic derivative product image are recognized, and a recognition result and an artistic value evaluation score are output. The recognition result includes fragmentation and integrity. The artistic value evaluation score has a value range of [0, 1] and is used to represent the influence of damage on the artistic value of the artistic derivative product.

[0041] If the recognition result is fragmentation, a fragmentation instruction is generated. If the recognition result is integrity, or the recognition result is fragmentation but the artistic value evaluation score is higher than a preset artistic value retention threshold, no fragmentation instruction is generated.

[0042] Further, the method of calculating the fragmentation area of the to-be-detected artistic derivative product comprises:

[0043] The area of the to-be-detected artistic derivative product is obtained. The area of the to-be-detected artistic derivative product is the area of the to-be-detected artistic derivative product in the analysis image corresponding to the fragmentation instruction.

[0044] The analysis image corresponding to the fragmentation instruction is read, the analysis image is subjected to grayscale processing, and the color image is converted into a grayscale image. The grayscale image is subjected to Gaussian filtering to reduce noise and details. The grayscale image subjected to the Gaussian filtering is subjected to edge detection to detect the edges of the to-be-detected artistic derivative product in the grayscale image. The grayscale image subjected to the edge detection is subjected to a binarization operation to convert the grayscale image into a binary image containing only edges and backgrounds. The contours in the binary image are searched. The searched contours are drawn on the corresponding analysis image. The area of the contours is calculated to obtain the area of the to-be-detected artistic derivative product.

[0045] The area of the to-be-detected artistic derivative product is subtracted from the area of the to-be-detected artistic derivative product to obtain the fragmentation area of the to-be-detected artistic derivative product.

[0046] Further, the method of obtaining the damage coefficient comprises:

[0047] When the to-be-detected artistic derivative product is produced on a historical production line, m to-be-detected artistic derivative product images are collected. The corresponding scratch degree and the fragmentation area of the to-be-detected artistic derivative product are analyzed for each of the m to-be-detected artistic derivative product images. Each to-be-detected artistic derivative product image corresponds to q scratch degrees and q fragmentation areas of the to-be-detected artistic derivative product. The q scratch degrees and the q fragmentation areas of the to-be-detected artistic derivative product corresponding to the m to-be-detected artistic derivative product images are sequentially added and then divided by m to obtain a first average value, that is, q first average values are obtained. Each first average value includes a first scratch degree average value and a fragmentation area average value of the to-be-detected artistic derivative product. The q groups of first average values are represented as a q x 2 matrix, which is denoted as , m is an integer greater than 0.

[0048] The second mean is calculated by summing the first mean values ​​of the q groups and then dividing by q. The second mean includes a second mean value for the scratches. The average fracture area of ​​an art derivative to be tested Marked as a matrix , ;

[0049] Subtracting the second mean from the first mean of each group yields a new matrix, denoted as... , ,matrix Dimensions and matrices same;

[0050] According to the matrix Calculate the covariance matrix ;

[0051] The degree of scratches and the area of ​​broken pieces of the art derivative to be detected corresponding to the i-th analysis image are labeled as a matrix. ,based on , and The distance to Maharanobis was calculated.

[0052] Set up a damage coefficient mapping table and obtain the basic damage coefficient of the corresponding analysis image based on the Mahalanobis distance;

[0053] The damage coefficient is obtained by weighting and summing the basic damage coefficient and the art value assessment score;

[0054] The method for setting up the damage coefficient mapping table is as follows:

[0055] The Mahalanobis distance is calculated based on the degree of scratches and the area of ​​breakage of multiple art derivatives under test that have scratches or breakage. Different damage coefficients are pre-set for the Mahalanobis distances corresponding to different art derivatives under test that have scratches or breakage, and the Mahalanobis distances correspond one-to-one with the damage coefficients.

[0056] The tolerance coefficient is obtained by weighting and summing the preset category price values ​​with the damage coefficient; the category price value is the value corresponding to the price of the art derivative storage object to be detected.

[0057] Preset tolerance threshold Tolerance coefficient With tolerance threshold To make a comparison, if Then the corresponding analyzed image will be marked as an abnormal image; if If so, the corresponding analyzed image will not be marked as an abnormal image;

[0058] The RCS robot is controlled to remove the art derivative products corresponding to abnormal images from the production line.

[0059] The intelligent vision-based detection system implements the aforementioned intelligent vision-based detection method, including:

[0060] The image acquisition module acquires images of the art derivatives to be detected, and these images are acquired according to the acquisition interval t.

[0061] The first image analysis module inputs the image of the art derivative to be detected into the trained scratch detection model, outputs the artistic annotation result, and determines whether to generate a scratch instruction with an AIGC art tag.

[0062] The scratch severity analysis module calculates the corresponding scratch severity based on the scratch command.

[0063] The second image analysis module inputs the image of the art derivative to be detected into the trained fragmentation analysis model, outputs the fragmentation recognition result and the art value assessment score, and determines whether to generate a fragmentation command.

[0064] The fracture degree analysis module calculates the fracture area of ​​the corresponding art derivative to be tested based on the fracture command.

[0065] The screening module combines the art value assessment score to obtain the corresponding damage coefficient, calculates the tolerance coefficient based on the damage coefficient and the art coefficient output by AIGC, and controls the RCS robot to remove the corresponding art derivative to be tested from the production line based on the tolerance coefficient.

[0066] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent vision-based detection method.

[0067] A computer-readable storage medium storing a computer program, which, when executed, implements the intelligent vision-based detection method.

[0068] The technical effects and advantages of the detection method, system, electronic device, and storage medium based on intelligent vision in this invention are as follows:

[0069] This invention targets art derivatives to be inspected. Images are acquired at calculated acquisition intervals (t). These images are then input into a trained scratch detection model to output artistic annotations, determining whether to generate a scratch instruction with an AIGC art tag and calculating the scratch severity. Simultaneously, the images are input into a trained fragmentation analysis model, which outputs fragmentation recognition results and an art value assessment score, determining whether to generate a fragmentation instruction and calculating the fragmentation area. A damage coefficient is obtained by combining the art value assessment score with the product category price value, and then normalized and weighted to obtain a tolerance coefficient. Based on the tolerance coefficient, an RCS robot is controlled to remove unqualified art derivatives from the production line. Specifically, during scratch detection model training, supplementary training images with the same artistic style as the images of the art derivatives to be inspected and containing multiple damage types are generated using AIGC. During fragmentation analysis model training, the art value assessment score is incorporated as an auxiliary feature, and the damage coefficient is obtained by a weighted sum of the base damage coefficient and the art value assessment score.

[0070] This invention solves the problems of low efficiency and error-proneness in traditional manual inspection, as well as the low efficiency, limited application scenarios, and inability to determine whether to remove the object under inspection based on the degree of damage in existing intelligent inspection systems, which require comprehensive image acquisition and single analysis. For art derivatives to be inspected, it also solves the problems of high misjudgment rate due to the inability of existing technologies to adapt to their visual artistic characteristics, loss of artistic value due to lack of artistic value consideration, poor detection generalization ability due to insufficient training data, and failure to meet the needs of review, traceability, or display due to the lack of artistic presentation of detection results. It realizes real-time intelligent inspection of art derivatives to be inspected in the production line, can quantitatively assess their quality problems, actively remove unqualified products, improve product quality and production efficiency, and at the same time take into account the control of artistic value, meeting the comprehensive needs of art derivative inspection scenarios. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the intelligent vision-based detection system according to Embodiment 1 of the present invention;

[0072] Figure 2 This is a schematic diagram of the analysis images in Embodiment 1 of the present invention;

[0073] Figure 3 This is a flowchart of the detection method based on intelligent vision according to Embodiment 2 of the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] Example 1

[0076] Please see Figure 1 As shown, the intelligent vision-based detection system described in this embodiment includes an image acquisition module, a first image analysis module, a scratch degree analysis module, a second image analysis module, a fragmentation degree analysis module, and a screening module; each module is connected via wired and / or wireless means to realize data transmission between modules.

[0077] The image acquisition module is used to acquire images of the art derivatives to be detected, such as containers, electronic products, and building materials; containers include bottles, boxes, and cans; electronic products include mobile phones and tablets; and building materials include tiles, glass windows, and metal doors and windows. In this embodiment, a bottle is used as an example, that is, the art derivative to be detected is a bottle, and the image acquisition module is used to acquire images of the bottle; the bottle images are acquired according to the acquisition interval t.

[0078] The bottle image is the image of the bottles on the production line. One bottle image contains n bottles, where n is an integer greater than 1. The bottle images are acquired by a high-speed camera installed on the RCS robot. The purpose of using a high-speed camera is that when the production line is running at a high speed, the high-speed camera can acquire bottle images more quickly than a regular camera, so as to achieve real-time and efficient detection.

[0079] Methods for obtaining the sampling interval t include:

[0080] The moving speed is obtained, which is the operating speed of the production line. The moving speed is calculated from the number of pulses, pulse duration, and resolution. The number of pulses is the number of pulse signals output by the encoder within the pulse duration, the pulse duration is the preset pulse signal acquisition time, and the resolution is the displacement corresponding to each pulse signal. Both the number of pulses and the resolution are obtained by the encoder installed on the production line's drive shaft, and the pulse signals are preset by those skilled in the art. The expression for the moving speed is: In the formula For movement speed, The number of pulses. For resolution, The pulse duration;

[0081] Length data is acquired, including the bottle diameter and the distance between bottles. The bottle diameter is the diameter of the bottom of the bottle, and the distance between adjacent bottles is the distance between them. The bottle distance is calculated based on the moving speed, the number of bottles collected, the collection time, and the bottle diameter. The number of bottles collected is a preset number, and the collection time is the time elapsed when the number of bottles collected by the photoelectric sensor reaches the preset number. The number of bottles collected is preset by those skilled in the art, and the collection time is acquired by a time sensor installed on the RCS robot. When the photoelectric sensor collects a bottle, the time sensor starts timing; when the number of bottles collected by the photoelectric sensor reaches the preset number, the time sensor stops timing. The data collected by the time sensor at this time is the collection time. The expression for the bottle distance is: In the formula, The distance between the bottles. For the time of collection, For the number of samples collected, The diameter of the bottle body is obtained by measuring the bottom of the bottle body using measurements taken by a person skilled in the art.

[0082] The data acquisition interval t is calculated based on the moving speed and length data. The expression for the data acquisition interval t is: .

[0083] The first image analysis module is used to analyze the bottle image and determine whether to generate a scratch instruction.

[0084] Methods for determining whether a scratch instruction has been generated include:

[0085] Step a. Input the bottle image into the trained scratch detection model, and label each of the n bottles in the bottle image with a rectangular box, that is, the n rectangular boxes correspond one-to-one with the n bottles;

[0086] Step b. Please refer to Figure 2 As shown, the bottle image containing the rectangular box annotation is labeled as the annotation image. The annotation image is cropped into n images according to the n rectangular boxes, and all n images are labeled as analysis images.

[0087] Step c. Perform grayscale processing on each of the n analysis images and determine whether there are scratches in the n analysis images; if scratches exist in the analysis images, generate a scratch instruction and mark the analysis image corresponding to the scratch instruction as a scratch image; if there are no scratches in the analysis images, do not generate a scratch instruction.

[0088] The specific training methods for the scratch detection model in step a include:

[0089] The bottle image is labeled as the first training image; supplementary training images with the same artistic style as the bottle image and various damage types and scratches are generated by AIGC; rectangular bounding boxes are pre-labeled for each of the n bottles in the first training image, and the n rectangular bounding boxes are sequentially labeled according to their positions in the first training image and the supplementary training image. , ... That is, marked as , Since the purpose of marking the rectangles is only to facilitate the differentiation of each rectangle, the order can be from right to left or from left to right. The first training images after labeling are divided into a training set and a test set. 70% of the first training images are used as the training set, and 30% are used as the test set. The scratch detection model is trained using the training set and tested using the test set. Preset coordinate error thresholds and art damage recognition accuracy thresholds are used. When the mean of the predicted coordinate errors of all first training images in the test set is less than the coordinate error threshold, and the art damage recognition accuracy is higher than the art damage recognition accuracy threshold, the scratch detection model training is complete, and the scratch detection model is output. The art damage recognition accuracy threshold is preset based on the category characteristics of the art derivative to be detected. Category characteristics include artistic style complexity and artistic value level, which are pre-evaluated by industry experts. The formula for calculating the mean of the predicted coordinate error includes:

[0090] ;

[0091] in To predict coordinate errors, The first training image is numbered. To determine the number of training images in the first training set, These are the vertices of the rectangle. Let x be the predicted x-coordinate of the k-th vertex of the i-th bounding box in the L-th first training image. Let x be the actual x-coordinate of the k-th vertex of the i-th rectangle in the L-th first training image. Let be the predicted ordinate of the k-th vertex of the i-th bounding box in the L-th first training image. The coordinate is the actual ordinate of the kth vertex of the i-th rectangle in the L-th first training image, where the rectangle has 4 vertices; the coordinate error threshold is preset according to the accuracy required by the scratch detection model.

[0092] The scratch detection model described above is specifically a convolutional neural network model;

[0093] The method for cropping the labeled image into n images based on n bounding boxes in step b includes:

[0094] Read the bottle image containing rectangular box annotations and the rectangular box coordinate file; extract the coordinates of the top left and bottom right corners of each rectangular box; based on the extracted coordinates of n rectangular boxes, crop n analysis images from the annotated image;

[0095] Specifically, you can use the `roi` and `Crop` functions in OpenCV. Here's a code example:

[0096] import cv2

[0097] img = cv2.imread('image.jpg')

[0098] for box in annotation:

[0099] x1, y1, x2, y2 = box['points']

[0100] roi = img[y1:y2, x1:x2]

[0101] cv2.imwrite(f'crop{i}.jpg', roi)

[0102] The methods for determining whether scratches exist in the n analyzed images in step c include:

[0103] Collect the grayscale values ​​of Y pixels corresponding to each analysis image, where Y is the total number of pixels in an analysis image;

[0104] Preset grayscale threshold , ; Compare the gray values ​​of the Y pixels corresponding to each analyzed image with the gray value threshold. If a comparative analysis is conducted, If so, then mark that pixel as a normal pixel. Then mark that pixel as a scratch point. Let be the grayscale value of the j-th pixel in the i-th image being analyzed. If a scratch point is present in an analysis image, then the corresponding analysis image is determined to have a scratch; if all points in an analysis image are normal, then the corresponding analysis image is determined to have no scratch.

[0105] It should be noted that the grayscale threshold... During the historical bottle scratch detection phase, a person skilled in the art acquires multiple images of bottles with scratches. These images are then bounded and cropped. The grayscale values ​​of the pixels corresponding to the scratches in each analysis image are used as an analysis set. The average of the highest grayscale values ​​across multiple analysis sets is used as the grayscale threshold. ;

[0106] It should be understood that a scratch on the bottle surface is a three-dimensional recessed area, which absorbs and reflects less light compared to the smooth bottle surface around the scratch. The gray value is the intensity of light reflected from the corresponding pixel to the camera, and the higher the gray value, the greater the light intensity. Since the scratched area absorbs and reflects less light, the corresponding gray value is lower than that of the smooth bottle surface around the scratch. Therefore, the gray value corresponding to the scratched area will be less than the gray value corresponding to the other smooth areas of the bottle surface.

[0107] The scratch severity analysis module calculates the corresponding scratch severity based on the scratch command.

[0108] The severity of scratches is calculated based on the area and number of scratches.

[0109] Methods for obtaining the scratch area include:

[0110] Count the number of scratch points and calculate the scratch area corresponding to each scratch image; the expression for scratch area is: In the formula Let be the area of ​​the scratch in the v-th scratch image. Let v be the number of scratch points in the v-th scratch image. The area of ​​a single pixel. This is the proportionality coefficient. W represents the number of scratch images;

[0111] The area of ​​a pixel is obtained from the resolution of the bottle image, which is obtained from the attributes of the bottle image. The actual area of ​​the bottle is divided by the area of ​​the bottle image as a proportionality coefficient. The actual area of ​​the bottle is the area corresponding to an unbroken bottle, and the area of ​​the bottle image is the image area corresponding to an unbroken bottle in the bottle image. Both the actual area of ​​the bottle and the area of ​​the bottle image are measured and obtained by those skilled in the art.

[0112] Methods for obtaining the number of scratches include:

[0113] Use NumPy to create W arrays with the same size as the scratch images as heatmaps, where each heatmap corresponds one-to-one with a scratch image. Initialize each element in each array to 0, thus constructing an all-zero matrix. The size of the scratch image is the width and height of the scratch image, which are obtained from the coordinates of the corresponding rectangle.

[0114] A Cartesian coordinate system is established with any vertex in each scratch image as the origin. The origin of each scratch image is the same vertex. The coordinates of each scratch point in each scratch image are obtained according to the Cartesian coordinate system. The corresponding heatmap array is indexed according to the coordinates of each scratch point in each scratch image, and the element at the corresponding position in the heatmap array is assigned a value. This embodiment is preferred. =1;

[0115] Apply the GaussianBlur function to each of the W thermal arrays to achieve Gaussian blur, specifying the Gaussian kernel size as . The standard deviation is The W arrays output after Gaussian blurring are used to overwrite the original thermal arrays and are marked as blurred arrays; this embodiment preferably uses... It is 15. =1;

[0116] Line segment detection algorithms (such as HoughTransform line segment detection) are used to detect line segments in W fuzzy arrays. The number of line segments detected in each fuzzy array is the number of scratches in the corresponding scratch image. Line segment detection algorithms are existing technology and will not be elaborated upon here. Specifically, the HoughLinesP function in OpenCV can be used, as shown in the code example below:

[0117] import cv2

[0118] import numpy as np

[0119] img = cv2.imread('image.jpg', 0)

[0120] lines = cv2.HoughLinesP(img, 1, np.pi / 180, 30,

[0121] minLineLength=50, maxLineGap=10)

[0122] for line in lines:

[0123] x1, y1, x2, y2 = line[0]

[0124] cv2.line(img, (x1, y1), (x2, y2), (0, 255, 0), 2)

[0125] cv2.imwrite('houghlines.jpg', img)

[0126] Methods for calculating the degree of scratches include:

[0127] ;

[0128] In the formula, Let v be the scratch severity of the v-th scratch image. Let v be the number of scratches in the v-th scratch image. , For preset weighting coefficients and , All are greater than 0;

[0129] The specific value of the weighting coefficient in the formula can be set according to the actual situation. The weighting coefficient reflects the degree of influence of the scratch area and the number of scratches on the scratch condition of the bottle surface. Those skilled in the art can preset the corresponding weighting coefficient according to the degree of influence of the scratch area and the number of scratches on the scratch condition of the bottle surface in order to accurately assess the scratch condition of the bottle surface.

[0130] It should be understood that the degree of scratch is the degree of influence of the scratch area and the number of scratches on the scratch situation of the bottle surface. The larger the scratch area, the more serious the scratch on the bottle surface, that is, the greater the degree of scratch, and vice versa. The more scratches there are, the more serious the scratch on the bottle surface, that is, the greater the degree of scratch, and vice versa. The degree of scratch is calculated in a dimensionless manner.

[0131] The second image analysis module is used to analyze the bottle image and determine whether a breakage command should be generated.

[0132] Methods for determining whether a fragmentation command has been generated include:

[0133] Using a trained fragmentation analysis model, identify n analysis images in the bottle image, and output the identification results and artistic value assessment score. The identification results include fragmentation and integrity.

[0134] The specific training process of the fragmentation analysis model includes:

[0135] Multiple analysis images are collected in advance, and each analysis image is labeled as a second training image. The bottle in each second training image is labeled, including whether it is broken or intact, and an artistic value assessment score is assigned to each analysis image. The artistic value assessment score is an indicator generated by AIGC technology to quantify the degree to which the possible broken or intact state of the bottle in the analysis image affects its artistic value. The calculation of this indicator is based on the integrity of the bottle's artistic style, the degree of integrity of the key artistic expression areas on the bottle's surface, and the market artistic value reference characteristics of bottles with similar artistic attributes. The concepts of "fragmented" and "intact" are converted into numerical labels; for example, "fragmented" is converted to 0 and "intact" to 1. The labeled second training images are divided into a training set and a test set, with 70% of the second training images used as the training set and 30% as the test set. When training the fragmentation analysis model using the training set, the artistic value assessment score is incorporated as an auxiliary feature into the training process. This allows the model to learn the correlation features between different states and the artistic value of the bottle while simultaneously learning to identify whether the bottle is fragmented or intact. This avoids the model focusing only on physical damage while ignoring the preservation of artistic value, thereby improving the accuracy of the model in identifying bottle damage states with artistic attributes. The fragmentation analysis model is then tested using the test set. A preset error threshold is used; when the mean prediction error of all second training images in the test set is less than the error threshold, the fragmentation analysis model is output. The formula for calculating the mean prediction error is... ,in For prediction error, This is the number of the second training image. For the first Predicted annotations for the second training image in group 2. For the first The actual annotations corresponding to the second training images in the group, where U is the number of second training images in the test set; the error threshold is preset according to the accuracy required by the fragmentation analysis model;

[0136] The aforementioned fragmentation analysis model is specifically a convolutional neural network model;

[0137] If the identification result is "broken", the system will determine whether to generate a breakage command based on the art value assessment score. If the art value assessment score is lower than the preset art value retention threshold, a breakage command will be generated, indicating that the bottle is broken and its art value has been damaged beyond acceptable limits. If the identification result is "intact", or if the identification result is "broken" but the art value assessment score is higher than the preset art value retention threshold, a breakage command will not be generated. The art value retention threshold is preset based on the industry standards of the art category to which the bottle belongs and the market's acceptance of the art value of this type of art bottle.

[0138] The fracture severity analysis module calculates the corresponding fracture area of ​​the bottle based on the fracture command.

[0139] The methods for calculating the area of ​​broken bottle body include:

[0140] Obtain the bottle area, which is the area of ​​the bottle in the analysis image corresponding to the breakage command;

[0141] The system reads the analysis image corresponding to the shattering command, converts the color image to grayscale, applies Gaussian filtering to the grayscale image to reduce noise and detail, performs edge detection on the grayscale image after Gaussian filtering to detect the edges of the bottle, binarizes the edge-detected grayscale image to convert it into a binary image containing only edges and background, uses the findContours function to find contours in the binary image, uses the drawContours function to draw the found contours on the corresponding analysis image for visualization, and calculates the area of ​​the contours to obtain the area of ​​the bottle.

[0142] Subtract the bottle area from the bottle image area to obtain the broken area of ​​the bottle.

[0143] The screening module obtains the corresponding damage coefficient based on the degree of scratches and the area of ​​bottle breakage, then calculates the corresponding tolerance coefficient based on the damage coefficient, and controls the RCS robot to remove the corresponding bottle from the production line based on the tolerance coefficient.

[0144] Methods for obtaining the damage coefficient include:

[0145] During bottle production on a historical production line, m bottle images were collected. The degree of scratches and the area of ​​bottle breakage were analyzed for each of the m images. Each bottle image corresponds to q degrees of scratches and q areas of bottle breakage. The q degrees of scratches and q areas of bottle breakage corresponding to the m bottle images were successively added together and then divided by m to obtain the first mean, i.e., q first means were obtained. Each first mean includes a first mean of scratch degree and a bottle breakage area. The q sets of first means were represented as a q×2 matrix, labeled as... m is an integer greater than 0; for example, the degree of the first scratch in the first bottle image is added to the degree of the first scratch in the second bottle image and the degree of the first scratch in the third bottle image, and so on, up to the degree of the first scratch in the m-th bottle image, and then divided by m to get the mean value corresponding to the degree of the first scratch; and so on to get the mean value corresponding to each of the q degrees of scratch.

[0146] The second mean is calculated by summing the first mean values ​​of the q groups and then dividing by q. The second mean includes a second mean value for the scratches. and the average area of ​​a broken bottle Marked as a matrix , For example, the scratch severity in the first mean of the first group is added to the scratch severity in the first mean of the second group and the scratch severity in the first mean of the third group, and so on, until the scratch severity in the first mean of the m-th group is obtained. Then, it is divided by q to get the second mean corresponding to the scratch severity.

[0147] Subtracting the second mean from the first mean of each group yields a new matrix, denoted as... , ,matrix Dimensions and matrices same;

[0148] According to the matrix Calculate the covariance matrix ;

[0149] ;

[0150] In the formula, Representation matrix transpose;

[0151] The methods for calculating the Mahalanobis distance include:

[0152] The degree of scratches corresponding to the i-th analysis image and the area of ​​bottle breakage Marked as a matrix ;

[0153] ;

[0154] In the formula, Denotes the Mahalanobis distance of the i-th analyzed image. It is the inverse of the covariance matrix;

[0155] Set up a damage coefficient mapping table and obtain the basic damage coefficient of the corresponding analysis image based on the Mahalanobis distance;

[0156] The damage coefficient mapping table is stored in the database. It can be a two-dimensional mapping table, which includes the Mahalanobis distance and the damage coefficient corresponding to the Mahalanobis distance. After calculating the Mahalanobis distance, the corresponding damage coefficient can be queried, which more intuitively reflects the damage situation on the bottle surface.

[0157] The method for setting up the damage coefficient mapping table is as follows:

[0158] In an experimental setting, based on various bottles with scratches or breakage encountered by those skilled in the art in their long-term work practice, Mahalanobis distances were calculated according to the degree of scratches and the area of ​​breakage of multiple bottles with scratches or breakage. Different damage coefficients were artificially set for the Mahalanobis distances corresponding to different bottles with scratches or breakage. The Mahalanobis distance and the damage coefficient correspond one-to-one. The larger the Mahalanobis distance, the more severe the damage to the bottle, and the larger the damage coefficient.

[0159] The damage coefficient is obtained by weighting and summing the basic damage coefficient and the art value assessment score. The specific process includes: pre-setting the weights of the basic damage coefficient and the art value assessment score according to the art category characteristics of the bottle, where the art category characteristics include the complexity of the art style, the level of art value, and the market acceptance of this type of art bottle; multiplying the basic damage coefficient by its corresponding weight to obtain the basic damage weighted value; multiplying the art value assessment score by its corresponding weight to obtain the art value assessment weighted value; and adding the basic damage weighted value and the art value assessment weighted value to obtain the damage coefficient.

[0160] The tolerance coefficient is obtained by weighting and summing preset category price values ​​with damage coefficients. Specific methods include:

[0161] The category price value corresponds to the price of the art derivative object to be detected.

[0162] Based on the category characteristics of the art derivatives to be tested, the weights of the category price value and the damage coefficient are preset. Category characteristics include artistic value level, market liquidity, and the degree of requirement for container integrity when storing the object. The category price value is normalized to obtain a category price index. The normalization method is to divide the category price value by the preset maximum category price value, so that the category price index is between 0 and 1. The damage coefficient is normalized to obtain a damage index. The normalization method is to divide the damage coefficient by the preset maximum damage coefficient, so that the damage index is between 0 and 1. The difference between 1 and the category price index is calculated to obtain the category price impact value. The difference between 1 and the damage index is calculated to obtain the damage impact value. The category price impact value is multiplied by the weight of the category price value to obtain the category price weighted value. The damage impact value is multiplied by the weight of the damage coefficient to obtain the damage weighted value. The category price weighted value and the damage weighted value are added to obtain the tolerance coefficient. This calculation process is dimensionless.

[0163] It should be understood that the tolerance coefficient is the degree tolerable for damage to the bottle. The larger the damage coefficient, the more serious the damage to the bottle, that is, the smaller the tolerance coefficient, and vice versa. The larger the category price value, the higher the price of the type of object stored in the bottle, and the higher the quality requirements for the bottle, that is, the smaller the tolerance coefficient. The tolerance coefficient is calculated in a dimensionless manner.

[0164] It should be noted that the product category price values ​​are pre-assigned by those skilled in the art based on the prices of the items stored in the bottles produced on the production line; higher prices are assigned higher values ​​to items with higher prices, and lower prices are assigned lower values ​​to items with lower prices.

[0165] Preset tolerance threshold Tolerance coefficient With tolerance threshold To make a comparison, if If the corresponding analysis image is marked as an abnormal image, it indicates that the bottle in the analysis image is severely damaged, the bottle is a defective product, and needs to be removed from the production line; if If the corresponding analysis image is not marked as an abnormal image, it means that the damage to the bottle in the analysis image is small and the bottle is a qualified product. The tolerance threshold is determined by a person skilled in the art to judge the quality of the bottles on the production line. The tolerance coefficients of multiple bottles judged as unqualified are calculated separately, and the smallest tolerance coefficient among the multiple tolerance coefficients is taken as the tolerance threshold. The bottles judged as unqualified are those with a large degree of damage.

[0166] Control the RCS robot to remove the bottle corresponding to the abnormal image from the production line;

[0167] This embodiment acquires images of bottles during production line operation, following a calculated acquisition interval. A trained model and algorithm are then used to detect scratches and breakage in the images, calculating the degree of scratches and the area of ​​breakage to quantitatively assess bottle damage. Based on the degree of damage, a tolerance coefficient is calculated, and this coefficient is used to control the RCS robot on the production line to remove bottles exceeding the tolerance range. This achieves real-time intelligent detection of bottles operating efficiently on the production line, enabling quantitative assessment of bottle quality issues and proactive removal of defective products based on damage severity, preventing low-quality products from flowing downstream and thus improving product quality and production efficiency.

[0168] Example 2

[0169] Please see Figure 3 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A detection method based on intelligent vision is provided, the method including:

[0170] Images of the art derivatives to be detected are collected, and these images are collected at intervals t.

[0171] Input the image of the art derivative to be detected into the trained scratch detection model, output the artistic annotation result, and determine whether to generate a scratch instruction with an AIGC art tag;

[0172] Calculate the corresponding scratch severity based on the scratch instruction;

[0173] Input the image of the art derivative to be detected into the trained fragmentation analysis model, output the fragmentation recognition result and art value assessment score, and determine whether to generate a fragmentation command.

[0174] Calculate the fracture area of ​​the corresponding art derivative to be tested based on the fracture command;

[0175] The damage coefficient is obtained by combining the art value assessment score. The tolerance coefficient is calculated based on the damage coefficient and the art coefficient output by AIGC. The corresponding tolerance coefficient is then calculated based on the damage coefficient. Based on the tolerance coefficient, the RCS robot is controlled to remove the corresponding art derivative to be tested from the production line.

[0176] Example 3

[0177] According to another aspect of this application, an electronic device is also provided. This electronic device may include one or more processors and one or more memories. The memories store computer-readable code that, when executed by the one or more processors, can perform the intelligent vision-based detection method described above.

[0178] Example 4

[0179] A computer-readable storage medium according to one embodiment of this application. The computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the intelligent vision-based detection method according to an embodiment of this application, as described with reference to the above figures, can be performed. The storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0180] Furthermore, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this application provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be executed by a processor to perform instructions corresponding to the method steps provided in this application, such as a detection method based on intelligent vision. When this computer program is executed by a central processing unit (CPU), it performs the functions defined in the method of this application.

[0181] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0182] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0183] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0184] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0185] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0187] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of inspection based on intelligent vision, characterized in that, The method comprises the following steps: Collecting an image of an art derivative product to be detected, wherein the image of the art derivative product to be detected is collected according to a collection interval t; Inputting the image of the art derivative product to be detected into a trained scratch detection model, performing rectangular frame labeling on n art derivative products to be detected in the image of the art derivative product to be detected, that is, n rectangular frames correspond to n art derivative products to be detected one by one, labeling the image of the art derivative product to be detected containing the rectangular frame labeling as a labeled image, cutting the labeled image into n images according to the n rectangular frames, marking the n images as analysis images, and determining whether to generate a scratch instruction accompanied by an AIGC art label; According to the scratch instruction, the corresponding scratch degree is calculated; Inputting the image of the art derivative product to be detected into a trained fragmentation analysis model, outputting a fragmentation recognition result and an artistic value evaluation score, and determining whether to generate a fragmentation instruction; According to the fragmentation instruction, the fragmentation area of the art derivative product to be detected is calculated; In the historical production line producing the to-be-detected artistic derivative, m times of to-be-detected artistic derivative images are collected, and the corresponding scratch degree and to-be-detected artistic derivative fragmentation area of each of the m to-be-detected artistic derivative images are analyzed, each to-be-detected artistic derivative image corresponding to q scratch degrees and q to-be-detected artistic derivative fragmentation areas; q scratch degrees and q to-be-detected artistic derivative fragmentation areas corresponding to the m to-be-detected artistic derivative images are sequentially added and then divided by m as a first average, that is, q first averages are obtained, each first average including a first scratch degree average and a to-be-detected artistic derivative fragmentation area average; q groups of first averages are expressed as a q×2 matrix, marked as , m is an integer greater than 0; The second average is calculated by adding the q sets of first averages in sequence and dividing by q and a second average of the scratch marks , labeled matrix , ; Subtracting the second mean from each of the first means results in a new matrix, denoted as , , which has the same dimensions as matrix . ​ According to the matrix Computing the covariance matrix ; The method for calculating the Mahalanobis distance comprises: The scratch degree corresponding to the i-th analysis image and the fragmentation area of the art derivative to be detected are marked as a matrix , based on , and Mahalanobis distance is calculated. Setting a damage coefficient mapping table, and obtaining a basic damage coefficient of the corresponding analysis image according to the Mahalanobis distance; Weighted sum of the basic damage coefficient and the artistic value evaluation score to obtain the damage coefficient; The setting method of the damage coefficient mapping table is: According to the Mahalanobis distance calculation of the scratch degree and the fragmentation area of the art derivative product to be detected with scratches or fragmentation, different damage coefficients are set for different Mahalanobis distances corresponding to different art derivative products to be detected with scratches or fragmentation, and the Mahalanobis distance and the damage coefficient correspond to each other; By weighted sum of the preset category price value and the damage coefficient, a tolerance coefficient is obtained; the category price value is a value corresponding to the price of the storage object of the art derivative product to be detected; Pre-set tolerance threshold , the tolerance coefficient is compared with the tolerance threshold , if , the corresponding analysis image is marked as an abnormal image; if , the corresponding analysis image is not marked as an abnormal image The RCS robot is controlled to move the art derivative product to be detected corresponding to the abnormal image out of the production line.

2. The smart vision based detection method of claim 1, wherein, The image of the art derivative product to be detected is the image of the art derivative product to be detected in the production line, and one image of the art derivative product to be detected includes n art derivative products to be detected, wherein n is an integer greater than 1; The method for obtaining the collection interval t comprises: The moving speed is obtained, and the moving speed is a running speed of the production line; the moving speed is obtained by pulse number, pulse time and resolution, the pulse number is a pulse signal number output by the encoder in the pulse time, the pulse time is a preset pulse signal collection time, and the resolution is a displacement corresponding to each pulse signal; an expression of the moving speed is: , wherein is the moving speed, is the pulse number, is the resolution, is the pulse time. Obtaining length data, the length data including an artistic derivative to be detected diameter and an artistic derivative to be detected interval, the artistic derivative to be detected diameter being a bottom diameter of the artistic derivative to be detected, and the artistic derivative to be detected interval being a distance between adjacent artistic derivatives to be detected; the artistic derivative to be detected interval being calculated by a moving speed, a collection quantity, a collection time and the artistic derivative to be detected diameter, the collection quantity being a preset collection quantity of the artistic derivative to be detected, the collection time being a time experienced when the collection quantity of the artistic derivative to be detected collected by the photoelectric sensor reaches the collection quantity; and an expression of the artistic derivative to be detected interval being: , wherein, is the artistic derivative to be detected interval, is the collection time, is the collection quantity, is the artistic derivative to be detected diameter. The acquisition interval t is calculated from the speed of movement and the length data, the expression for which is: .

3. The smart vision based detection method of claim 2, wherein, The method for determining whether to generate the scratch instruction comprises: Gray processing is performed on the n analysis images respectively, and it is determined whether there is a scratch in the n analysis images; if there is a scratch in the analysis image, a scratch instruction is generated, and the analysis image corresponding to the scratch instruction is marked as a scratch image; if there is no scratch in the analysis image, no scratch instruction is generated.

4. The smart vision based detection method of claim 3, wherein, The training method of the scratch detection model comprises: The image of the art derivative to be detected is marked as a first training image; a supplementary training image with multiple damage types and scratches is generated by AIGC, which is consistent with the artistic style of the image of the art derivative to be detected; the damage positions in the supplementary training image are labeled with rectangular boxes; the first training image and the supplementary training image are divided into a training set and a test set, the scratch detection model is trained using the training set, and the scratch detection model is tested using the test set; a preset coordinate error threshold and an artistic damage recognition accuracy threshold are set, when the average value of the predicted coordinate errors of all the first training images in the test set is less than the coordinate error threshold, and the artistic damage recognition accuracy is higher than the artistic damage recognition accuracy threshold, the scratch detection model is trained, and the scratch detection model is output; the scratch detection model is a convolutional neural network model.

5. The smart vision based detection method of claim 3, wherein, The method for cutting n images from a labeled image according to n rectangular boxes comprises: Read the image of the art derivative to be detected containing rectangular box labeling and the rectangular box coordinate file; extract the coordinates of the upper left corner and the lower right corner of each rectangular box, and cut n analysis images from the labeled image according to the coordinates corresponding to the n rectangular boxes.

6. The smart vision based detection method of claim 3, wherein, The method for determining whether there are scratches in the n analysis images comprises: Collecting the gray scale values of Y pixel points corresponding to each analysis image , Y is the number of all pixel points in one analysis image; is the gray scale value of the jth pixel point in the ith analysis image, ; Preset grayscale threshold , ;like If so, the corresponding pixel will be marked as a normal pixel. If a scratch point is found, the corresponding pixel is marked as a scratch point; if a scratch point exists in an analysis image, it is determined that a scratch exists in the corresponding analysis image; otherwise, it is determined that a scratch does not exist in the corresponding analysis image.

7. The smart vision based detection method of claim 1, wherein, The scratch degree is calculated and obtained according to the scratch area and the number of scratches; The method for obtaining the scratch area comprises: The number of scratch points is planned, and the scratch area corresponding to each scratch image is calculated; the expression of the scratch area is: , wherein is the scratch area of the vth scratch image, is the number of scratch points of the vth scratch image, is the area of one pixel point, is a proportional coefficient, , and W is the number of scratch images. An area of a pixel point is acquired from a resolution of the to-be-detected artistic derivative product image, and the resolution of the to-be-detected artistic derivative product image is acquired from an attribute of the to-be-detected artistic derivative product image; a ratio coefficient is obtained by dividing an actual area of the to-be-detected artistic derivative product by an area of the to-be-detected artistic derivative product image , the actual area of the to-be-detected artistic derivative product is an area corresponding to an undamaged to-be-detected artistic derivative product, and the area of the to-be-detected artistic derivative product image is an image area corresponding to an undamaged to-be-detected artistic derivative product in the to-be-detected artistic derivative product image The method for obtaining the number of scratches comprises: Create W arrays consistent with the size of the scratch image as heat arrays, wherein the heat arrays correspond to the scratch images one by one, and a zero matrix is constructed; the size of the scratch image is the width and height of the scratch image; A rectangular coordinate system is established with any vertex in each scratch image as the origin, and the origins of each scratch image are the same vertex. Coordinates of each scratch point in each scratch image are obtained according to the rectangular coordinate system; and elements at corresponding positions in the thermal force array are assigned values of according to the coordinates of each scratch point in each scratch image. Gaussian blur is performed on the W thermal arrays, a Gaussian kernel size of and a standard deviation of are specified; the W arrays output after Gaussian blur are respectively overlaid on the original thermal arrays and are marked as blurred arrays; Use a line segment detection algorithm to detect line segments in the W fuzzy arrays, and the number of line segments detected in each fuzzy array is the number of scratches of the corresponding scratch image; The scratch degree is obtained by weighted summation of the scratch area and the number of scratches of the scratch image.

8. The smart vision based detection method of claim 7, wherein, The method for determining whether to generate a fragmentation instruction comprises: Use the trained fragmentation analysis model to identify the n analysis images in the image of the art derivative to be detected, and output the identification result and the artistic value evaluation score; the identification result includes fragmentation and integrity; the artistic value evaluation score ranges from 0 to 1, and is used to represent the influence of damage on the artistic value of the art derivative; If the identification result is fragmentation, a fragmentation instruction is generated; if the identification result is integrity, or the identification result is fragmentation but the artistic value evaluation score is higher than the preset artistic value retention threshold, no fragmentation instruction is generated.

9. The smart vision based detection method of claim 1, wherein, The method for calculating the fragmentation area of the art derivative to be detected comprises: Obtain the area of the art derivative to be detected, which is the area of the art derivative to be detected in the analysis image corresponding to the fragmentation instruction; The reading fragmentation instruction corresponds to the analysis image, the analysis image is subjected to gray scale processing, and the color image is converted into a gray scale image; the gray scale image is subjected to Gaussian filtering to reduce noise and details; the gray scale image subjected to the Gaussian filtering is subjected to edge detection to detect the edges of the art derivative to be detected in the gray scale image; the gray scale image subjected to the edge detection is subjected to a binaryzation operation to convert the gray scale image into a binary image containing only edges and backgrounds; the contours in the binary image are searched; the searched contours are drawn on the corresponding analysis image; the area of the contours is calculated to obtain the area of the art derivative to be detected; The area of the art derivative to be detected is subtracted from the area of the art derivative to be detected to obtain the fragmentation area of the art derivative to be detected.

10. A smart vision based inspection system implementing the smart vision based inspection method of any one of claims 1-9, characterized in that, Comprise: An image acquisition module acquires an art derivative to be detected image, and the art derivative to be detected image is acquired according to an acquisition interval t; A first image analysis module inputs the art derivative to be detected image into a trained scratch detection model, outputs an artistic labeling result, and judges whether a scratch instruction with an AIGC artistic label is generated; A scratch degree analysis module calculates the corresponding scratch degree according to the scratch instruction; A second image analysis module inputs the art derivative to be detected image into a trained fragmentation analysis model, outputs a fragmentation recognition result and an artistic value evaluation score, and judges whether a fragmentation instruction is generated; A fragmentation degree analysis module calculates the fragmentation area of the art derivative to be detected corresponding to the fragmentation instruction; A screening module combines the artistic value evaluation score to obtain a corresponding damage coefficient, calculates a tolerance coefficient according to the damage coefficient and an artistic coefficient output by AIGC, and controls the RCS robot to remove the corresponding art derivative to be detected in the production line according to the tolerance coefficient.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the intelligent vision-based detection method of any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed to realize the intelligent vision-based detection method of any one of claims 1-9.

Images