Data collection method and system for a station unit

Abstract

The application relates to a data collection method and system of a work station unit. By receiving a first image, picture information in the first image can be extracted, the picture information including one or more of text information, digital information and product shape information. The picture information is compared with a target of production inspection, so that the time and human resources of production inspection can be saved, and a preliminary process of production inspection of the work station unit is completed. Since a picture modification software is common at present, in order to improve the authenticity of the first image and perfect the process of production inspection of the work station unit, by receiving a second image, the authenticity of shooting of the first image can be identified by using the second image. The authenticity of the information of the first image can be verified. Meanwhile, the second image can have content information of a photographer of the first image, so based on a three-dimensional image restoration algorithm, the authenticity of the second image itself can be determined.

Description

Technical Field

[0001] This application relates to the field of image algorithm technology, and in particular to a data collection method and system for a workstation unit. Background Technology

[0002] High-efficiency and high-quality production is an important way for factories to pursue economic benefits. To achieve high-efficiency production, factories often adopt assembly line work methods, where each workstation is independent. To achieve high-quality production, factories often use inspection methods to conduct production checks on each workstation.

[0003] Traditional production inspection methods at workstations are time-consuming and manpower-intensive. Completing the production inspection process at workstations while saving time and manpower remains a challenge for the industry. Therefore, it is necessary to propose a data collection method and system for workstations to address the resource-intensive nature of traditional workstation-based production inspections. Summary of the Invention

[0004] Therefore, it is necessary to propose a data collection method and system for workstations to address the shortcomings of traditional workstation-based production inspection, which is relatively resource-intensive.

[0005] This application provides a data collection method for a workstation unit, including:

[0006] Receive a first image and a second image; the content of the first image is the production content, and the content of the second image is the content of the first image and the actions of the photographer when the first image was captured.

[0007] Based on a 3D image reconstruction algorithm, perspective feature extraction of the second image is achieved;

[0008] Use an image recognition algorithm to extract the content of the first image from the second image;

[0009] Determine whether the content of the first image in the second image matches the target of the production inspection;

[0010] If the content of the first image in the second image does not match the target of the production inspection, a prompt message indicating an abnormality in the inspection process will be provided.

[0011] If the content of the first image in the second image matches the target of the production inspection, then it is further determined whether the perspective features of the second image match the three-dimensional image features in the three-dimensional image reconstruction algorithm;

[0012] If the perspective features of the second image do not match the 3D image features in the 3D image reconstruction algorithm, a message indicating an error has occurred in the inspection process will be sent.

[0013] If the perspective features of the second image match the 3D image features in the 3D image reconstruction algorithm, a message indicating that the inspection process is complete will be displayed.

[0014] This application provides a data collection system for a workstation unit, including:

[0015] The processor is used to execute the data collection method of the workstation unit.

[0016] The first image acquisition device is communicatively connected to the processor;

[0017] The second image acquisition device is communicatively connected to the processor.

[0018] This application relates to a data collection method and system for a workstation unit. By receiving a first image, image information can be extracted from the first image, including one or more types of text information, numerical information, and product appearance information. Comparing the image information with the production inspection target can complete the preliminary production inspection process of the workstation unit while saving production inspection time and manpower. Since image modification software is currently quite common, to improve the authenticity of the first image and refine the production inspection process of the workstation unit, a second image can be received to verify the authenticity of the first image. Specifically, the second image contains the work order image and the photographer of the first image from the first image. The work order image in the second image can verify the authenticity of the information in the first image. Simultaneously, the second image may contain information about the photographer of the first image; therefore, based on a 3D image reconstruction algorithm, the authenticity of the second image itself can be determined. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a data collection method for a workstation unit according to an embodiment of this application.

[0020] Figure 2 This is a structural connection diagram of a data collection system for a workstation unit provided in an embodiment of this application.

[0021] Figure label:

[0022] 100 - Processor; 200 - First image acquisition unit; 300 - Second image acquisition unit. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] This application provides a data collection method for a workstation unit.

[0025] like Figure 1 As shown in one embodiment of this application, a data collection method for a workstation unit includes:

[0026] S100, receive the first image and the second image. The content of the first image is the production content, and the content of the second image is the content of the first image and the actions of the photographer when capturing the first image.

[0027] Specifically, the content of the first image mainly consists of the work order and product to be inspected. Production inspection can be carried out by utilizing the work order content and product appearance image information in the first image.

[0028] For example, taking a photo of a project work order yields an image of the work order, while taking a photo of a product yields a 3D image; both types of images can be used as the first image.

[0029] The content of the second image is mainly the content of the first image and the actions of the photographer when taking the first image.

[0030] For example, photographer A takes a picture of a product with camera A1, obtaining the first image. Photographer B, holding camera B1, takes a picture of photographer A's action while photographer A is taking the picture with camera A1, obtaining the second image. Therefore, the second image must contain both the content of the first image (i.e., what photographer A photographed) and the act of taking the picture (i.e., the action of the photographer in taking the first image).

[0031] It should be noted that the content of the first image contained in the second image does not need to be of excessively high resolution, as long as it can be identified in subsequent steps.

[0032] S200, based on a 3D image reconstruction algorithm, extracts perspective features from the second image.

[0033] Specifically, since the content of the first image can be the work order of the project to be inspected, it is difficult to determine whether the first image is a real-time captured image through a 3D image reconstruction algorithm under this working condition.

[0034] Therefore, it is necessary to review the correctness of the content of the first image by using the content of the first image and the second image taken by the photographer of the first image, and determine the authenticity of the second image based on the real-time three-dimensional spatial state of the photographer of the first image.

[0035] Because the second image covers more content, relying solely on the project work order content and product appearance information from the first image within the second image makes image recognition more difficult and results in lower accuracy. By using the second image to verify the project work order content and product appearance information from the first image, the error tolerance rate of the entire workstation unit's production inspection process is higher.

[0036] S300 invokes an image recognition algorithm to extract the content of the first image from the second image.

[0037] Specifically, image recognition algorithms can be used to extract the content of the first image and to extract the content of the first image from the second image.

[0038] Since the first image contains text, when text appears, image recognition algorithms, such as Convolutional Neural Networks (CNNs), can be used to automatically learn and extract the text from the first image by constructing a deep network structure.

[0039] When product appearance images are available, the K-means clustering algorithm can be used for image segmentation. It can be used without pre-labeled training data to obtain product appearance image information.

[0040] S400, determine whether the content of the first image in the second image matches the target of the production inspection.

[0041] S500: If the content of the first image in the second image does not match the target of the production inspection, a prompt message indicating an abnormality in the inspection process will be fed back.

[0042] S600, if the content of the first image in the second image matches the target of the production inspection, then further determine whether the perspective features of the second image match the three-dimensional image features in the three-dimensional image restoration algorithm.

[0043] S700, if the perspective features of the second image do not match the three-dimensional image features in the three-dimensional image reconstruction algorithm, a prompt message indicating an abnormality in the inspection process will be fed back.

[0044] S800, if the perspective features of the second image match the three-dimensional image features in the three-dimensional image reconstruction algorithm, then a prompt message indicating that the inspection process has been completed is returned.

[0045] This embodiment relates to a data collection method for a workstation unit. By receiving a first image, image information can be extracted from the first image. This image information includes one or more of text information, numerical information, and product appearance information. Comparing the image information with the production inspection target can complete the initial production inspection process of the workstation unit while saving production inspection time and manpower. Since image modification software is currently quite common, to improve the authenticity of the first image and refine the production inspection process of the workstation unit, a second image can be received. The authenticity of the first image can be verified using the second image. Specifically, the second image contains the work order image and the photographer of the first image. The work order image in the second image can verify the authenticity of the information in the first image. Simultaneously, the second image may contain information about the photographer of the first image; therefore, based on a 3D image reconstruction algorithm, the authenticity of the second image itself can be determined.

[0046] In one embodiment of this application, S100 includes:

[0047] S111, Receive the first image and the second image.

[0048] Specifically, the first image and the second image are captured simultaneously to ensure that the content of the second image has three-dimensional spatial characteristics.

[0049] S112, perform illumination correction on the first image and the second image respectively to obtain the corrected first image and the corrected second image.

[0050] Specifically, image preprocessing is a series of steps designed to improve the quality of image data and provide more accurate input data for subsequent image analysis and pattern recognition processes. These processing methods include illumination correction, blurring and denoising, image pyramids, edge extraction and enhancement, etc.

[0051] S113, Gaussian filtering is applied to the corrected first image and the corrected second image respectively to obtain the first initial image and the second initial image.

[0052] Specifically, by using the weights of the Gaussian function to perform a weighted average of neighboring pixels, normally distributed noise can be effectively removed, and this method is widely used in the noise reduction process in image processing.

[0053] S114, binarize the first initial image to obtain a grayscale image of the first image.

[0054] Specifically, the image is converted into a binary image with only two grayscale values ​​for image segmentation and object extraction. The binary grayscale values ​​can be used for edge extraction and enhancement, and can subsequently be used to divide the grayscale image into blocks in a two-dimensional plane.

[0055] S115, binarize the second initial image to obtain the grayscale image of the second image.

[0056] This embodiment relates to image preprocessing, a crucial component of computer vision tasks. Preprocessing improves image quality through techniques such as illumination correction, blurring and denoising, image pyramiding, and edge extraction and enhancement, making the image more suitable for further analysis and processing. In this embodiment, appropriate preprocessing methods can be selected based on the specific task, such as text recognition or product geometric appearance recognition, to ensure the accuracy of the final result.

[0057] More specifically, when text appears, image recognition algorithms, such as Convolutional Neural Networks (CNNs), can be used to automatically learn and extract the text appearing in the first image by constructing a deep network structure. In this case, preprocessing methods such as blurring and denoising can improve the accuracy of text recognition.

[0058] When product appearance images are available, the K-means clustering algorithm can be used for image segmentation, without requiring pre-labeled training data, to obtain the product appearance image information. In this case, preprocessing methods such as edge extraction and enhancement can improve the accuracy of the product appearance image information.

[0059] In one embodiment of this application, S100 further includes:

[0060] S121, perform edge extraction on the grayscale image to select multiple core information blocks from the grayscale image.

[0061] Specifically, the image is converted into a binary image with only two grayscale values: white and gray, for image segmentation and object extraction. Binary grayscale values ​​can be used for edge extraction and enhancement, allowing for block edge extraction in a two-dimensional plane. In this edge extraction method, regions with grayscale values ​​have distinct edges from pure white regions, and the core information blocks are the grayscale extraction objects. In fact, grayscale images can also be segmented hierarchically according to grayscale values, with each grayscale layer connected by vectors. Hierarchical segmentation adds a dimension to the two-dimensional space—the grayscale value. Layered segmented grayscale images can then be used for edge extraction and enhancement in a three-dimensional data system.

[0062] S122, Select a core information block.

[0063] Specifically, the segmented grayscale image is a two-dimensional grayscale image c.

[0064] S123, based on the VGG deep learning neural network, extracts information from each core information block.

[0065] Specifically, the information in the core information block includes one or more of text information, numerical information, and product appearance information.

[0066] S123a, Establish a filter for the grayscale image and use the grayscale image as the original image to be convolved.

[0067] Specifically, a color / grayscale image has multiple RGB channels, and multiple filters with the same number of channels can be used to convolve the image.

[0068] S123b generates a secondary convolutional image based on the filter's stride.

[0069] Specifically, the distance the filter moves parallel to the image to be convolved is the filter's step size.

[0070] The filter can move parallel to the image to be convolved in two directions. The segmented grayscale image is a grayscale image in a two-dimensional plane.

[0071] S123c uses a pooler to pool the secondary convolutional image.

[0072] Specifically, poolers can be implemented using either max pooling or average pooling. Poolers do not participate in convolution calculations; they are primarily used for dimensionality reduction of the image to be convolved.

[0073] S123d, return the filter used to create the grayscale image until the dimension of the secondary convolutional image reaches the dimension threshold.

[0074] S123e performs a full connection on the secondary convolutional images that have reached the dimensionality threshold.

[0075] Specifically, the fully connected process can actually be a matrix representation of the convolution result of the previous level image to be convolved, and based on the matrix representation result, the next level image to be convolved can be generated.

[0076] S123f obtains information about the core information block in the grayscale image.

[0077] Specifically, by using the results of multiple matrix transformations, the dimensionality of the original image to be convolved can be reduced to a level that is output.

[0078] More specifically, information can be obtained from grayscale images by outputting highly reliable results.

[0079] S124, return to the step of selecting a core information block, until all core information blocks have been selected.

[0080] In one embodiment of this application, S200 includes:

[0081] S211, Based on the grayscale image of the second image, outline the grayscale image of the second image to obtain multiple outline images.

[0082] Specifically, the Sobel operator detects contours by calculating the gradient of gray values ​​around each pixel in the image. It uses two 3x3 kernels to convolve the original image in the horizontal and vertical directions, respectively, to obtain two components of the gradient, and finally merges these two components to obtain the edge image.

[0083] S212, Select a profile.

[0084] S213, the perspective lines that form the outline of the drawing.

[0085] Specifically:

[0086] S213a, calls a second image.

[0087] S213b, based on the division of gray values, forms a multi-layer grayscale image.

[0088] S213c convolves the grayscale image of each layer in both the horizontal and vertical directions.

[0089] S213d yields the two components of the gradient in the horizontal and vertical directions.

[0090] S213e, based on the grayscale image of each layer, merges the two components of the gradient in the horizontal and vertical directions.

[0091] S213f, obtain the edge image.

[0092] S213g defines the edge image as a set of contour maps of the second image.

[0093] Specifically, grayscale images can also be segmented hierarchically according to grayscale values, with each grayscale layer connected by vectors. Hierarchical segmentation adds a dimension to the two-dimensional space—the grayscale value. Layered grayscale images can then undergo edge extraction and enhancement within a three-dimensional data system.

[0094] S213z, Select one contour map from the set of contour maps of the second image.

[0095] S213y extracts multiple line segments from the contour map.

[0096] S213x, extend each straight line segment to form the perspective lines of the outline.

[0097] S213w, return one of the contour maps in the set of contour maps selected for the second image, until all contour maps have been selected.

[0098] Specifically, the Sobel operator emphasizes edges, especially areas with significant grayscale changes, by calculating the gradient of each pixel in the image. By setting an appropriate threshold, straight line segments can be extracted from the Sobel-processed image.

[0099] It is worth mentioning that in two-dimensional space, straight lines parallel to the two-dimensional space base coordinates will be eliminated, and the remaining straight line segments can be used to determine the focal point of the perspective line.

[0100] In current image acquisition devices, the focal points of perspective lines may not be unique, but the spacing between these focal points is specific. The spacing between the focal points of perspective lines is one of the 3D image features in 3D image reconstruction algorithms.

[0101] S214, return to the previous step of selecting a contour map, until all contour maps have been selected.

[0102] S221, Select a perspective line for a contour plot.

[0103] S222, Based on the perspective lines of the outline, determine the focal point of the perspective lines.

[0104] S223 defines the focal point of the perspective line as a feature of the second image.

[0105] S224, return to the point where you selected a perspective line of a contour map, until all perspective lines of the contour maps have been selected.

[0106] Specifically, in images captured in real-time by an image acquisition device, the focal point of the perspective lines reflects the actual three-dimensional spatial position of objects within the image. Modified images, however, struggle to achieve the same level of accuracy in reflecting the actual three-dimensional spatial position of objects through the focal point of their perspective lines.

[0107] Simply put, the focal point of the perspective line in a real-time captured image cannot coincide with the focal point of the perspective line in a modified image.

[0108] In one embodiment of this application, S600 includes:

[0109] S611, Select a grayscale image of a second image.

[0110] S612, determine the geometric center of the grayscale image of the second image in the two-dimensional plane, and take the geometric center of the grayscale image of the second image in the two-dimensional plane as a three-dimensional image feature.

[0111] S613, Select a focal point of a perspective line.

[0112] S614, determine whether the focal point of the perspective line coincides with the geometric center of the second image.

[0113] S615, if the focal point of the perspective line does not coincide with the geometric center of the second image, then it is determined that the features of the second image do not match the features of the three-dimensional image in the three-dimensional image restoration algorithm, and the process returns to selecting a grayscale image of the second image until all grayscale images of the second images have been selected.

[0114] S616, if the focal point of the perspective line coincides with the geometric center of the second image, then return to the step of selecting a focal point of a perspective line, until the focal points of all perspective lines have been selected.

[0115] S617, when all the focal points of the perspective lines have been selected, determine whether all the focal points of the selected grayscale image of the second image coincide with the geometric center of the second image.

[0116] S618, if the focal points of all perspective lines of the selected grayscale image of the second image coincide with the geometric center of the second image, then it is determined that the features of the second image match the features of the three-dimensional image in the three-dimensional image restoration algorithm, and the process returns to selecting a grayscale image of the second image until all grayscale images of the second images have been selected.

[0117] S619, if the focal point of at least one perspective line does not coincide with the geometric center of the second image, then it is determined that the features of the second image do not match the features of the three-dimensional image in the three-dimensional image restoration algorithm, and the process returns to selecting a grayscale image of the second image until all grayscale images of the second images have been selected.

[0118] This application provides a data collection system for a workstation unit.

[0119] like Figure 2 As shown, in one embodiment of this application, a data collection system for a workstation unit includes a processor 100, a first image acquisition unit 200, and a second image acquisition unit 300.

[0120] The processor 100 is used to execute the data collection method of the workstation unit.

[0121] The first image acquisition device 200 is communicatively connected to the processor 100.

[0122] The second image acquisition unit 300 is communicatively connected to the processor 100.

[0123] This embodiment relates to a data collection system for a workstation unit. The processor 100 receives a first image captured by a first image acquisition device 200 and extracts image information from it. This image information includes one or more of text, numerical, and product shape information. Comparing the image information with the production inspection target can complete the initial production inspection process of the workstation unit while saving time and manpower. Since image modification software is currently common, to improve the authenticity of the first image and refine the production inspection process of the workstation unit, a second image captured by a second image acquisition device 300 is received. The authenticity of the first image can be verified using the second image. Specifically, the second image contains information such as the work order image and the photographer of the first image. The work order image in the second image can verify the authenticity of the information in the first image. Simultaneously, the second image may contain information about the photographer of the first image; therefore, based on a three-dimensional image reconstruction algorithm, the authenticity of the second image itself can be determined.

[0124] The technical features of the above embodiments can be combined arbitrarily, and the execution order of the method steps is not restricted. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data collection method for a workstation unit, characterized in that, include: Receive a first image and a second image; the content of the first image is the production content, and the content of the second image is the content of the first image and the actions of the photographer when the first image was captured. Based on a 3D image reconstruction algorithm, perspective feature extraction of the second image is achieved; Use an image recognition algorithm to extract the content of the first image from the second image; Determine whether the content of the first image in the second image matches the target of the production inspection; If the content of the first image in the second image does not match the target of the production inspection, a prompt message indicating an abnormality in the inspection process will be provided. If the content of the first image in the second image matches the target of the production inspection, then determine whether the perspective features of the second image match the three-dimensional image features in the three-dimensional image reconstruction algorithm. If the perspective features of the second image do not match the 3D image features in the 3D image reconstruction algorithm, a message indicating an error has occurred in the inspection process will be sent. If the perspective features of the second image match the 3D image features in the 3D image reconstruction algorithm, a message indicating that the inspection process has been completed will be sent. The three-dimensional image reconstruction algorithm for extracting perspective features from the second image includes: Based on the grayscale image of the second image, outlines are drawn on the grayscale image of the second image to obtain multiple outline images. Select a silhouette image; The perspective lines that form the outline of the drawing; Return to the previous step and select a contour map, until all contour maps have been selected; The method for extracting perspective features from the second image based on the 3D image reconstruction algorithm also includes: Select a perspective line for the outline; Based on the perspective lines of the outline, determine the focal point of the perspective lines; Define the focal point of the perspective line as a perspective feature of the second image; Return to the previous step and select a perspective line of the outline until all perspective lines of the outlines have been selected; Select a grayscale image of a second image; Determine the geometric center of the grayscale image of the second image in the two-dimensional plane, and use the geometric center of the grayscale image of the second image in the two-dimensional plane as a three-dimensional image feature.

2. The data collection method for a workstation unit according to claim 1, characterized in that, Receiving the first image and the second image includes: Receive the first image and the second image; Illumination correction is performed on the first image and the second image respectively to obtain the corrected first image and the corrected second image; The corrected first image and the corrected second image are subjected to Gaussian filtering respectively to obtain the first initial image and the second initial image; The first initial image is binarized to obtain a grayscale image of the first image; The second initial image is binarized to obtain a grayscale image of the second image.

3. The data collection method for a workstation unit according to claim 2, characterized in that, The receiving of the first image and the second image also includes: Edge extraction is performed on the grayscale image to filter out multiple core information blocks from the grayscale image; Select a core information block; Based on the VGG deep learning neural network, information is extracted from each core information block; the information in the core information block includes one or more of text information, numerical information, and product appearance information; Return to the previous step and select a core information block until all core information blocks have been selected.

4. The data collection method for a workstation unit according to claim 3, characterized in that, The information extracted from each core information block based on the VGG deep learning neural network includes: Establish a filter for the grayscale image and use the grayscale image as the original convolutional image; Based on the stride of the filter, a secondary convolutional image is generated; Pool the secondary convolutional image using a pooler; Return to the filter used to create the grayscale image until the dimension of the secondary convolutional image reaches the dimension threshold; Fully connect the secondary convolutional images that reach the dimensionality threshold; Obtain information about the core information blocks in the grayscale image.

5. The data collection method for a workstation unit according to claim 4, characterized in that, The perspective lines forming the outline include: Call a second image; Based on the division of gray values, multi-layer grayscale images are formed; Convolve the grayscale image of each layer in both the horizontal and vertical directions; Obtain the two components of the gradient in the horizontal and vertical directions; Based on the grayscale image of each layer, the two components of the gradient in the horizontal and vertical directions are merged. Obtain the edge image; The edge image is defined as a set of contour maps of the second image.

6. The data collection method for a workstation unit according to claim 5, characterized in that, The perspective lines forming the outline also include: Select one contour map from the set of contour maps for the second image; Extract multiple line segments from the contour map; Extend each straight line segment to form the perspective lines of the outline; Return to one of the contour maps in the set of selected second images, until all contour maps have been selected.

7. The data collection method for a workstation unit according to claim 6, characterized in that, The step of determining whether the features of the second image match the features of the three-dimensional image in the three-dimensional image reconstruction algorithm includes: Select a focal point of the perspective line; Determine whether the focal point of the perspective line coincides with the geometric center of the second image; If the focal point of the perspective line does not coincide with the geometric center of the second image, it is determined that the features of the second image do not match the features of the three-dimensional image in the three-dimensional image restoration algorithm, and the process returns to selecting a grayscale image of the second image until all grayscale images of the second images have been selected.

8. The data collection method for a workstation unit according to claim 7, characterized in that, The step of determining whether the features of the second image match the features of the three-dimensional image in the three-dimensional image reconstruction algorithm also includes: If the focal point of the perspective line coincides with the geometric center of the second image, then return to the step of selecting a focal point of a perspective line, until the focal points of all perspective lines have been selected; When all the focal points of the perspective lines have been selected, determine whether all the focal points of the selected grayscale image of the second image coincide with the geometric center of the second image; If the focal points of all perspective lines of the selected grayscale image of the second image coincide with the geometric center of the second image, then it is determined that the features of the second image match the three-dimensional image in the three-dimensional image restoration algorithm, and the process of selecting a grayscale image of the second image is repeated until all grayscale images of the second images have been selected. If the focal point of at least one perspective line does not coincide with the geometric center of the second image, it is determined that the features of the second image do not match the three-dimensional image in the three-dimensional image restoration algorithm, and the process returns to selecting a grayscale image of the second image until all grayscale images of the second images have been selected.

Images