Pin product deformation detection method by expanding the field of view

Abstract

The present invention discloses a method for detecting deformation of pin products by expanding the field of view, which relates to the technical field of image processing and includes: collecting an initial product image within an initial field of view; dividing the product image to obtain a plurality of grid initial images within a plurality of grids, respectively performing deformation recognition on them to obtain a plurality of initial deformation information; when there is deformation in the plurality of initial deformation information, predicting the deformation trend of the plurality of edge grid initial images to obtain a plurality of deformation trend information; combining the plurality of edge initial deformation information and the plurality of deformation trend information, calculating to obtain a field of view expansion coefficient, adjusting the initial field of view range to obtain an expanded field of view range, continuing to perform deformation detection on the target pin products, and combining the plurality of initial deformation information to obtain a final deformation detection result. The present invention solves the technical problems of limited deformation detection range and insufficient detection accuracy in the prior art, and achieves the technical effect of improving the deformation detection accuracy and comprehensiveness of the target pin products.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for detecting deformation of pin products by expanding the field of view. Background Art

[0002] With the continuous development of modern manufacturing, especially in the production of precision electronic components and micro-devices, the quality control of pin products has become increasingly important. Traditional methods for detecting deformation of pin products are usually limited by the detection range and field of view, making it difficult to comprehensively and accurately capture the deformation of the products. Such detection methods often only cover the central part of the product, and the edges and hard-to-reach areas are easily overlooked, resulting in incomplete deformation detection and thus affecting the quality control of the products. Summary of the Invention

[0003] The present application provides a method for detecting deformation of pin products by expanding the field of view, which is used to solve the technical problems of limited deformation detection range and insufficient detection accuracy in the prior art.

[0004] In view of the above problems, the present application provides a method for detecting deformation of pin products by expanding the field of view.

[0005] The present application provides a method for detecting deformation of pin products by expanding the field of view, and the method includes:

[0006] Collecting an initial product image within the initial field of view of a target pin product to be detected by a depth perception camera; dividing the product image according to a grid division rule to obtain a plurality of grid initial images within a plurality of grids, respectively performing deformation recognition to obtain a plurality of initial deformation information; when deformation exists in the plurality of initial deformation information, predicting the deformation trend of the plurality of edge grid initial images according to the plurality of edge initial deformation information of the plurality of edge grids to obtain a plurality of deformation trend information; combining the plurality of edge initial deformation information and the plurality of deformation trend information, calculating to obtain a field of view expansion coefficient, adjusting the initial field of view range to obtain an expanded field of view range, continuing to perform deformation detection on the target pin product, and combining the plurality of initial deformation information to obtain a final deformation detection result.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] In this application, an initial product image within the initial field of view of the target pin product to be detected is collected by a depth perception camera; according to the grid division rule, the product image is divided to obtain multiple grid initial images within multiple grids, and deformation recognition is performed on each of them to obtain multiple initial deformation information; when deformation exists in the multiple initial deformation information, based on the multiple edge initial deformation information of multiple edge grids, deformation trend prediction is performed on the multiple edge grid initial images to obtain multiple deformation trend information; by combining the multiple edge initial deformation information and the multiple deformation trend information, a field of view expansion coefficient is calculated, the initial field of view is adjusted to obtain an expanded field of view, and the deformation detection of the target pin product is continued. By combining the multiple initial deformation information, a final deformation detection result is obtained. The present invention solves the technical problems of limited deformation detection range and insufficient detection accuracy in the prior art. By collecting the initial image of the target pin product through a depth perception camera, performing deformation recognition according to grid division to obtain initial deformation information, predicting the deformation trend based on the edge grid deformation information, calculating the field of view expansion coefficient, adjusting the field of view, and obtaining more deformation information, the technical effects of improving the deformation detection accuracy and comprehensiveness of the target pin product are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 It is a schematic flowchart of a method for detecting deformation of a pin product by expanding the field of view provided by an embodiment of this application;

[0011] Figure 2 It is a schematic flowchart of obtaining multiple deformation trend information in the method for detecting deformation of a pin product by expanding the field of view provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] This application provides a method for detecting deformation of a pin product by expanding the field of view, which is used to solve the technical problems of limited deformation detection range and insufficient detection accuracy in the prior art. By collecting the initial image of the target pin product through a depth perception camera, performing deformation recognition according to grid division to obtain initial deformation information, predicting the deformation trend based on the edge grid deformation information, calculating the field of view expansion coefficient, adjusting the field of view, and obtaining more deformation information, the technical effects of improving the deformation detection accuracy and comprehensiveness of the target pin product are achieved.

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0014] It should be noted that any variations of the terms "including" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0015] Embodiment, such as Figure 1 As shown, the present application provides a method for detecting the deformation of a pin product by expanding the field of view. The method includes:

[0016] Step S100: Collect an initial product image within the initial field of view of the target pin product to be detected through a depth perception camera.

[0017] Furthermore, the method provided by the embodiments of the application further includes:

[0018] The depth perception camera is a 2.5D camera.

[0019] In the embodiments of the present application, first, a depth perception camera is used to collect images of the target pin product to be detected. Among them, the depth perception camera is a 2.5D camera, which can simultaneously obtain the two-dimensional image and depth information of the target product. The 2.5D camera collects an image containing the depth data of each pixel point through its built-in depth perception technology, thereby providing richer spatial information. The initial field of view of the depth perception camera is determined by parameters such as the focal length and viewing angle preset by humans. The depth perception camera is used to collect images of the target pin product to be detected, and an initial product image within the initial field of view is obtained.

[0020] Step S200: Divide the product image according to the grid division rule to obtain multiple grid initial images within multiple grids, and perform deformation recognition respectively to obtain multiple initial deformation information.

[0021] In the embodiments of the present application, when dividing the product image, first divide the product image into multiple grids of the same size according to the grid division rule, including edge grids and central grids. Each grid contains a part of the image area, and the grid size is preset.

[0022] Subsequently, perform deformation recognition on the initial images within these grids, determine whether each grid has undergone deformation, and obtain the initial deformation information of each grid. The initial deformation information is the ratio of the deformation size to the product size.

[0023] Furthermore, in the method provided by the application embodiment, according to the grid division rule, divide the product image to obtain multiple initial grid images within multiple grids, perform deformation recognition on each respectively, and obtain multiple initial deformation information. It further includes:

[0024] According to the grid division rule, divide the product image to obtain multiple initial grid images within multiple grids. Among them, the grid division rule includes multiple grids of the same size, and the multiple grids include edge grids and central grids; perform deformation recognition on the multiple initial grid images to obtain multiple initial deformation information.

[0025] In the embodiment of the present application, when performing deformation detection on the target pin product, first process the collected initial product image. According to the grid division rule, divide the image into multiple grid regions of the same size. This grid size is preset to ensure that each grid can effectively cover the key information in the image while maintaining an appropriate computational complexity. The grid division includes edge grids and central grids, where the edge grids are located around the image and may contain interference factors; while the central grids are located in the middle area of the image and contain more detailed information, which is the key area for deformation recognition.

[0026] After completing the grid division, perform deformation recognition on the initial images within each grid. Specifically, first, based on the historical deformation detection data of the pin product, collect and process the sample initial grid image set, and label the deformation size ratio within each sample grid image to generate the initial deformation information set of the samples. Then, use these sample initial images and deformation information labeled with deformation information as supervised training data, and use a convolutional neural network (CNN) for training to learn the relationship between the image and the deformation. After training, input the multiple initial grid images into the trained deformation recognition path, and output the initial deformation information of each grid after recognition.

[0027] Furthermore, in the method provided by the application embodiment, when performing deformation recognition on the multiple initial grid images to obtain multiple initial deformation information, it further includes:

[0028] According to the historical deformation detection data of the pin products, collect and process to obtain a set of initial grid images of the samples, and identify the size ratio of the deformation of the pin products in each initial grid image of the sample to obtain a set of initial deformation information of the samples; use the set of initial grid images of the samples and the set of initial deformation information of the samples as supervised training data, and based on the convolutional neural network, train the deformation recognition path; input the multiple initial grid images into the deformation recognition path, and identify and output to obtain multiple initial deformation information.

[0029] In the embodiment of the present application, first obtain the historical deformation detection data of the pin products from the historical database, and then divide the product images in the historical deformation detection data into multiple small pieces according to the grid division rule, and each small piece is an initial grid image. Through this process, a set of initial grid images of the samples is obtained.

[0030] Next, identify the size ratio of the deformation of the pin products in each initial grid image of the sample. This process is carried out in the way of manual annotation, and the annotation process includes calculating the ratio of the deformation size to the original size of the product. For example, if there is a protrusion in the pin product in the grid, and the height of the protrusion is 0.1 mm and the thickness of the pin is 1 mm, then the size ratio of the deformation is 10%. Through this process, a set of initial deformation information of the samples is obtained.

[0031] Subsequently, use the set of initial grid images of the samples and the set of initial deformation information of the samples as supervised training data, and train through the convolutional neural network. The convolutional neural network is a deep learning algorithm, which is good at automatically extracting features from images and performing classification or regression tasks. During the training process, the network continuously adjusts the parameters to learn how to identify the deformation patterns and features of the pin products from the images. After training, the CNN can automatically judge and output the corresponding deformation information according to the input grid images. After the training is completed, the deformation recognition path is obtained.

[0032] Finally, input the multiple initial grid images into the trained deformation recognition path, and the model analyzes according to the learned features and outputs the initial deformation information of each grid. Through this process, multiple initial deformation information is obtained.

[0033] Step S300: When there is deformation in the multiple initial deformation information, predict the deformation trend of the multiple initial grid images of the multiple edge grids according to the multiple edge initial deformation information of the multiple edge grids to obtain multiple deformation trend information.

[0034] In an embodiment of the present application, when there is deformation among multiple initial deformation information, first, the deformation information of multiple edge grids and the corresponding initial images of the edge grids are screened out. Then, a deformation trend prediction path is constructed based on the ensemble learning method, which includes multiple deformation prediction branches. After that, the ratio of each initial edge deformation information to the maximum value among all the initial edge deformation information is calculated, and after multiplying by the total number of deformation prediction branches and taking the integer, the number of deformation prediction branches is obtained. Then, the multiple initial images of the edge grids are input into the corresponding number of deformation prediction branches for deformation trend prediction. Finally, by calculating the mean value of the multiple predicted deformation trend information, the final multiple deformation trend information is obtained.

[0035] Further, as Figure 2 shown, in the method provided by the embodiment of the application, when there is deformation among the multiple initial deformation information, based on the multiple initial edge deformation information of multiple edge grids, the deformation trend of the multiple initial images of the edge grids is predicted to obtain multiple deformation trend information, and it further includes:

[0036] When any one of the multiple initial deformation information is greater than 0, the multiple initial edge deformation information of multiple edge grids and the multiple initial images of the edge grids are screened out; based on ensemble learning, a deformation trend prediction branch including the total number of deformation prediction branches is constructed to obtain a deformation trend prediction path; the ratio of each initial edge deformation information to the maximum value among the multiple initial edge deformation information is calculated respectively, and after multiplying by the total number of deformation prediction branches and taking the integer respectively, the number of multiple deformation prediction branches is obtained; the multiple initial images of the edge grids are respectively input into the deformation trend prediction branches with the number of the multiple deformation prediction branches, and the prediction output obtains a set of multiple predicted deformation trend information; the mean value of the set of multiple predicted deformation trend information is calculated respectively to obtain multiple deformation trend information.

[0037] In an embodiment of the present application, when any one of the multiple initial deformation information is greater than 0, the multiple initial edge deformation information of the multiple edge grids obtained by the foregoing division and the multiple initial images of the edge grids are obtained.

[0038] Next, based on ensemble learning, according to the deformation detection data of historical pin products, an initial image set of sample edge grids is collected, and a set of sample deformation trend information is identified. The deformation trend information includes the deformation probability of each sample, which is determined according to the proportion of deformation in the area of the sample edge grid initial image far from the central grid area. Then, the sample images and deformation trend information are divided into multiple supervised training data sets for constructing deformation trend prediction branches including the total number of deformation prediction branches. Through an ensemble-based convolutional neural network (CNN), multiple deformation prediction branches are trained respectively, and finally a deformation trend prediction path is obtained. Among them, the total number of deformation prediction branches is preset.

[0039] Then, calculate the ratio of each initial edge deformation information to the maximum value among multiple initial edge deformation information respectively. This ratio reflects the severity of the deformation of each edge grid relative to other grids. Next, multiply this ratio by the total number of deformation prediction branches and take the integer to obtain an integer value, which represents the number of deformation prediction branches corresponding to each edge grid. Through this process, multiple numbers of deformation prediction branches are obtained.

[0040] After obtaining the number of deformation prediction branches, input the initial images of multiple edge grids into the corresponding number of deformation prediction branches for processing, and each branch will independently process the input image and output the prediction result. Among them, the deformation prediction branches are randomly selected from the deformation trend prediction path according to the number of deformation prediction branches. Through this process, multiple sets of predicted deformation trend information are obtained.

[0041] Finally, by calculating the mean value of multiple sets of predicted deformation trend information respectively, multiple final deformation trend information is obtained.

[0042] Furthermore, in the method provided by the application embodiment, based on ensemble learning, constructing deformation trend prediction branches including the total number of deformation prediction branches and obtaining a deformation trend prediction path further includes:

[0043] According to the deformation detection data of pin products within a historical time, an initial image set of sample edge grids is collected, and a set of sample deformation trend information is identified according to the proportion of deformation in the area of different sample edge grid initial images far from the central grid direction. Among them, each sample deformation trend information includes the sample deformation probability; divide the initial image set of the sample edge grids and the set of sample deformation trend information to obtain multiple copies of supervised training data of the total number of deformation prediction branches; use the multiple copies of supervised training data, based on an ensemble convolutional neural network, train the deformation trend prediction branches of the total number of deformation prediction branches respectively, and after the training is completed, obtain a deformation trend prediction path.

[0044] In the embodiment of the present application, first, deformation detection data is collected from the historical database to obtain an initial image set of sample edge grids. The historical database contains image data collected during the past deformation detection of pin products, recording the deformation conditions of pin products at different time periods. By analyzing these historical data, an initial image set of sample edge grids is obtained.

[0045] Next, according to the deformation ratio of the regions of different initial images of sample edge grids away from the central grid direction, a set of sample deformation trend information is obtained by identification. Among them, the region away from the central grid direction refers to the part outside the image edge, that is, the region not captured by the camera. This part is manually detected, and the deformation information is artificially stored in the historical database. Different initial images of sample edge grids refer to the initial images of sample edge grids with different initial deformation information. According to the deformation ratio of the regions of different initial images of sample edge grids away from the central grid direction, that is, the ratio obtained by calculating the number of deformed regions of the initial images of sample edge grids with the same initial deformation information away from the central grid direction divided by the number of the initial images of sample edge grids with the same initial deformation information, as the sample deformation trend information, that is, the sample deformation probability. Then, according to the calculated ratio, the initial image set of sample edge grids is identified to obtain a set of sample deformation trend information.

[0046] Subsequently, the initial image set of sample edge grids and the set of sample deformation trend information are divided. The specific method is to evenly divide these data into multiple pieces of supervised training data according to the total number of deformation prediction branches. Each piece of data includes an edge grid image and the corresponding deformation trend information, which is used to train the model.

[0047] Finally, multiple pieces of supervised training data are used to train the deformation trend prediction branches with the total number of deformation prediction branches respectively based on the integrated convolutional neural network. Through this process, multiple deformation trend prediction branches with the total number of deformation prediction branches are trained respectively, and the final deformation trend prediction path is obtained through the trained network model, which is used to predict the deformation trend of new samples.

[0048] Step S400: Combine the multiple initial deformation information and the multiple deformation trend information, calculate to obtain a field of view expansion coefficient, adjust the initial field of view range to obtain an expanded field of view range, continue to perform the deformation detection of the target pin product, and combine the multiple initial deformation information to obtain a final deformation detection result.

[0049] In the embodiments of the present application, first, by calculating the ratio of each initial edge deformation information to the mean of multiple initial edge deformation information respectively, a plurality of deformation trend correction coefficients are obtained. Then, these correction coefficients are respectively applied to a plurality of deformation trend information to obtain a plurality of corrected deformation trend information. By calculating the mean of these corrected deformation trend information, a field of view expansion coefficient is obtained. Using this field of view expansion coefficient to adjust the initial field of view range, an expanded field of view range is obtained.

[0050] Then, through a depth perception camera, according to the expanded field of view range, an expanded product image of the target pin product is collected. The area of the initial product image in the expanded product image is cropped off, and the area outside the initial product image in the expanded product image is retained for deformation recognition to obtain the deformation information in the expanded range. Finally, the deformation information in the expanded range is combined with a plurality of initial deformation information to obtain the final deformation detection result.

[0051] Further, in the method provided by the application embodiments, when combining the plurality of initial edge deformation information and a plurality of deformation trend information to calculate and obtain a field of view expansion coefficient and adjust the initial field of view range to obtain an expanded field of view range, it further includes:

[0052] Calculating the ratio of each initial edge deformation information to the mean of the plurality of initial edge deformation information respectively to obtain a plurality of deformation trend correction coefficients; respectively multiplying the plurality of deformation trend correction coefficients by the plurality of deformation trend information to obtain a plurality of corrected deformation trend information; calculating the mean of the plurality of corrected deformation trend information to obtain a field of view expansion coefficient; multiplying the initial field of view range by the field of view expansion coefficient to adjust and obtain an expanded field of view range.

[0053] In the embodiments of the present application, first, calculate the ratio of each initial edge deformation information to the mean of the plurality of initial edge deformation information to obtain a plurality of deformation trend correction coefficients.

[0054] Next, use the obtained plurality of deformation trend correction coefficients to correct the corresponding plurality of deformation trend information. Specifically, multiply each correction coefficient by its corresponding deformation trend information to obtain a plurality of corrected deformation trend information.

[0055] After obtaining a plurality of corrected deformation trend information, calculate the mean of the plurality of corrected deformation trend information, and use the calculation result as the field of view expansion coefficient.

[0056] Subsequently, multiply the initial field of view range by the field of view expansion coefficient to adjust and obtain an expanded field of view range.

[0057] Further, in the method provided by the application embodiments, when continuing the deformation detection of the target pin product and combining the plurality of initial deformation information to obtain the final deformation detection result, it further includes:

[0058] Through a depth perception camera, according to the expanded field of view range, an expanded product image of the target pin product is collected; the product image of the area outside the initial product image in the expanded product image is cropped, and deformation recognition is performed to obtain expanded range deformation information; the final deformation detection result is obtained by combining the expanded range deformation information and the multiple initial deformation information.

[0059] In the embodiment of the present application, first, an expanded product image of the target pin product is collected through a depth perception camera according to the expanded field of view range calculated above.

[0060] Next, the product image of the area outside the initial product image in the expanded product image is cropped. Specifically, the collected expanded product image is cropped to retain the area outside the initial product image. This means that the area covered by the original product image is removed from the expanded product image, and the newly added area beyond the initial field of view is retained.

[0061] Then, deformation recognition is performed on the newly added area after cropping. Specifically, the newly added area after cropping is divided according to the grid division rule to obtain multiple grid images, which are input into the deformation recognition path trained above to obtain expanded range deformation information.

[0062] Finally, the expanded range deformation information obtained in the newly added area is combined with the multiple initial deformation information within the initial field of view to obtain the final deformation detection result of the target pin product.

[0063] In the embodiment of the present application, in summary, the embodiment of the present application has at least the following technical effects:

[0064] In this application, an initial product image within the initial field of view of a target pin product to be detected is collected by a depth perception camera; according to a grid division rule, the product image is divided to obtain multiple grid initial images within multiple grids, and deformation recognition is performed on each of them to obtain multiple initial deformation information; when there is deformation in the multiple initial deformation information, based on the multiple edge initial deformation information of multiple edge grids, deformation trend prediction is performed on the multiple edge grid initial images to obtain multiple deformation trend information; combining the multiple edge initial deformation information and multiple deformation trend information, a field of view expansion coefficient is calculated, the initial field of view is adjusted to obtain an expanded field of view, and the deformation detection of the target pin product is continued. Combining the multiple initial deformation information, a final deformation detection result is obtained. The present invention solves the technical problems of limited deformation detection range and insufficient detection accuracy in the prior art. By collecting the initial image of the target pin product through a depth perception camera, performing deformation recognition according to grid division to obtain initial deformation information, predicting the deformation trend based on the edge grid deformation information, calculating the field of view expansion coefficient, adjusting the field of view, and obtaining more deformation information, the technical effects of improving the deformation detection accuracy and comprehensiveness of the target pin product are achieved.

[0065] It should be noted that the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above has described specific embodiments of this specification. The processes depicted in the drawings do not necessarily require the specific order and continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0066] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

[0067] This specification and the drawings are only exemplary descriptions of the present application and are considered to have covered any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. A method for detecting deformation of a pin product by expanding the field of view, characterized in that: The method comprises: Using a depth perception camera, an initial product image within an initial field of view of the target pin product to be inspected is collected; According to the grid division rule, the product image is divided to obtain a plurality of grid initial images in a plurality of grids, and deformation recognition is performed respectively to obtain a plurality of initial deformation information, including: Dividing the product image according to a grid division rule to obtain a plurality of grid initial images in a plurality of grids, wherein the grid division rule includes a plurality of grids of the same size, and the plurality of grids include edge grids and a central grid; Performing deformation recognition on the multiple grid initial images to obtain multiple initial deformation information; When the multiple initial deformation information are deformed, a deformation trend prediction is performed on the multiple edge grid initial images according to the multiple edge initial deformation information of the multiple edge grids to obtain multiple deformation trend information; Combining the multiple edge initial deformation information and the multiple deformation trend information, calculating and obtaining the field of view expansion coefficient, adjusting the initial field of view range to obtain the expanded field of view range, continuing to perform deformation detection of the target pin product, and combining the multiple initial deformation information to obtain the final deformation detection result; The combining the plurality of edge initial deformation information and the plurality of deformation trend information to calculate the field of view expansion coefficient, adjusting the initial field of view range to obtain the expanded field of view range, comprises: Respectively calculating the ratio of each edge initial deformation information to the average of the plurality of edge initial deformation information to obtain a plurality of deformation trend correction coefficients; The plurality of deformation trend correction coefficients are respectively used to multiply the plurality of deformation trend information to obtain a plurality of corrected deformation trend information; Calculating the average of the plurality of modified deformation trend information to obtain a field of view expansion coefficient; The field of view expansion coefficient is multiplied by the initial field of view to adjust and obtain the expanded field of view.

2. The method for detecting pin product deformation by expanding the field of view according to claim 1, characterized in that: The depth perception camera is a 2.5D camera.

3. The method for detecting pin product deformation by expanding the field of view according to claim 1, characterized in that: Performing deformation recognition on the multiple grid initial images to obtain multiple initial deformation information includes: According to the historical deformation detection data of the pin product, a sample initial raster image set is collected and processed, and the size ratio of the pin product deformation in each sample initial raster image is marked to obtain a sample initial deformation information set; Using the sample initial grid image set and the sample initial deformation information set as supervised training data, and training a deformation recognition path based on a convolutional neural network; The multiple grid initial images are input into the deformation recognition path, and the recognition output is used to obtain multiple initial deformation information.

4. The method for detecting pin product deformation by expanding the field of view according to claim 1, characterized in that: When the multiple initial deformation information is deformed, deformation trend prediction is performed on the multiple edge grid initial images according to the multiple edge initial deformation information of the multiple edge grids to obtain multiple deformation trend information, including: When any one of the plurality of initial deformation information is greater than 0, screening a plurality of edge initial deformation information of a plurality of edge grids and a plurality of edge grid initial images; Based on ensemble learning, a deformation trend prediction branch including the total number of deformation prediction branches is constructed to obtain a deformation trend prediction path; Calculating the ratio of each edge initial deformation information to the maximum value of the multiple edge initial deformation information respectively, multiplying by the total number of deformation prediction branches respectively and rounding, to obtain multiple numbers of deformation prediction branches; Inputting the plurality of edge grid initial images into the deformation trend prediction branches of the plurality of deformation prediction branches respectively, and outputting the predictions to obtain a plurality of predicted deformation trend information sets; The average values ​​of the plurality of predicted deformation trend information sets are calculated respectively to obtain a plurality of deformation trend information.

5. The method for detecting pin product deformation by expanding the field of view according to claim 4, characterized in that: Based on ensemble learning, a deformation trend prediction branch including the total number of deformation prediction branches is constructed to obtain a deformation trend prediction path, including: According to the deformation detection data of the pin products in the historical time, a set of sample edge grid initial images is collected, and according to the proportion of deformation in the area away from the central grid direction of different sample edge grid initial images, a set of sample deformation trend information is obtained by marking, wherein each sample deformation trend information includes the sample deformation probability; Dividing the sample edge grid initial image set and the sample deformation trend information set to obtain multiple supervised training data of the total deformation prediction branch number; The plurality of supervised training data are used to train deformation trend prediction branches of the total number of deformation prediction branches based on an integrated convolutional neural network, and after the training is completed, a deformation trend prediction path is obtained.

6. The method for detecting pin product deformation by expanding the field of view according to claim 1, characterized in that: Continuing to perform deformation detection of the target pin product, combining the multiple initial deformation information to obtain a final deformation detection result, including: By using a depth perception camera, an enlarged product image of the target pin product is collected according to the enlarged field of view; cropping the product image in the area outside the initial product image within the enlarged product image, performing deformation recognition, and obtaining deformation information of the enlarged range; The expanded range deformation information and the multiple initial deformation information are combined to obtain a final deformation detection result.

Images