Image processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides an image processing method and device, a storage medium and a program product, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first image group; acquiring a plurality of second image groups; based on a set mark area of each first image in the first image group, performing consistency scoring on a mark corresponding area of each second image in each second image group to obtain a local consistency score; based on each first image in the first image group, performing consistency scoring on each second image in each second image group to obtain a global consistency score; and determining a target consistency score according to the local consistency score and the global consistency score, and determining a second image group meeting a set consistency score requirement and a second set imaging parameter corresponding to the second image group based on the target consistency score. According to the scheme, the image consistency in the machine copying process can be ensured, and a foundation is laid for accurately completing a detection task aiming at the target object subsequently.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, storage medium and program product. Background Art

[0002] In the practical application of machine vision imaging, high-quality imaging is crucial for subsequent visual inspection and analysis tasks. However, in actual production, "machine cloning" is often required: imaging the same product using the same hardware (light source, lens, camera) as the original machine. However, imaging of the same product on different machines can be affected by factors such as the production environment, machine differences, and hardware aging. This can lead to inconsistencies in texture, brightness, contrast, and color, increasing the complexity and error rate of subsequent inspection algorithms. Therefore, there is an urgent need for an image processing method that can ensure image consistency during this "machine cloning" process.

[0003] Among them, "machine" specifically refers to an independent, complete and deployable visual imaging detection unit, which usually includes hardware (such as light source, lens, camera, etc.), mechanical structure (such as precision slide, rotary table, fixture and other positioning structures, as well as dust cover, shock absorption platform, temperature control box and other protective structures) and control system (such as industrial computer that runs parameter control software and image processing algorithm). Summary of the Invention

[0004] Multiple aspects of the present application provide an image processing method, device, storage medium and program product to ensure the consistency of images during the "machine replication" process, laying the foundation for the subsequent accurate completion of detection tasks for target objects.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: Acquiring a first image group obtained by capturing an image of a target object using a first machine based on first set imaging parameters, wherein the first machine includes a hardware device for performing image capture; Acquiring a plurality of second image groups obtained by respectively capturing images of the target object using a second machine based on a plurality of second set imaging parameters, wherein the second machine comprises the same hardware device as the first machine, the plurality of second set imaging parameters being obtained by adjusting the first set imaging parameters by setting adjustment ranges, and the second set imaging parameters and the second image groups having a one-to-one correspondence; Based on the set marked area of ​​each first image in the first image group, performing a consistency score on the marked corresponding area of ​​each second image in each second image group to obtain a local consistency score, the marked corresponding area corresponding to the set marked area; performing a consistency score on each second image in each second image group based on each first image in the first image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image; A target consistency score is determined according to the local consistency score and the global consistency score, so as to determine a second image group that meets a set consistency score requirement and second set imaging parameters corresponding to the second image group based on the target consistency score.

[0006] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising: An acquisition module is configured to acquire a first image group obtained by acquiring images of a target object using a first machine based on first set imaging parameters, the first machine including hardware for image acquisition, and to acquire a plurality of second image groups obtained by acquiring images of the target object using a second machine based on a plurality of second set imaging parameters, the second machine including the same hardware as the first machine, the plurality of second set imaging parameters being obtained by adjusting the first set imaging parameters by setting an adjustment range, and the second set imaging parameters corresponding one-to-one to the second image groups.

[0007] A scoring module is configured to perform a consistency score on the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group to obtain a local consistency score, where the marked corresponding area corresponds to the set marked area; and to perform a consistency score on each second image in each second image group based on each first image in the first image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image.

[0008] A determination module is configured to determine a target consistency score based on the local consistency score and the global consistency score, so as to determine a second image group that meets the set consistency score requirement and second set imaging parameters corresponding to the second image group based on the target consistency score.

[0009] In a third aspect, an embodiment of the present application provides an electronic device comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the method described in the first aspect.

[0010] In a fourth aspect, an embodiment of the present application provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the method described in the first aspect.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it can implement the method described in the first aspect.

[0012] In an embodiment of the present application, the acquisition of all images requiring consistency scoring is achieved by acquiring a first image set obtained by capturing images of a target object using a first machine based on first set imaging parameters, and multiple second image sets obtained by capturing images of the target object using a second machine based on multiple second set imaging parameters. Local consistency scores are obtained by performing consistency scoring on the marked areas corresponding to the marks in each second image in each second image set based on the set marked areas of each first image in the first image set. A global consistency score is obtained by performing consistency scoring on each second image in each second image set based on each first image in the first image set. This ensures that the resulting target consistency score comprehensively considers both local and global factors, overcomes the impact of hardware differences and environmental fluctuations on the score, and ensures the accuracy of the target consistency score. Subsequently, based on the more accurate target consistency score, a second image set and the corresponding second set imaging parameters are determined that meet the set consistency score requirements. This ensures image consistency during the "machine replication" process, laying the foundation for the accurate completion of subsequent inspection tasks such as defect detection and size detection of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of an image processing method provided by an exemplary embodiment of the present application; Figure 2 A flowchart of determining a marked corresponding area of ​​each second image in each second image group provided by an exemplary embodiment of the present application; Figure 3 A schematic diagram of an application for determining a marking corresponding area provided by an exemplary embodiment of the present application; Figure 4 A schematic diagram of an application of geometric alignment processing of image blocks provided by an exemplary embodiment of the present application; Figure 5 A specific example diagram of geometric alignment processing of image blocks provided by an exemplary embodiment of the present application; Figure 6 A schematic diagram of an application for determining a local consistency score provided by an exemplary embodiment of the present application; Figure 7 A schematic diagram of an application for determining a global consistency score provided by an exemplary embodiment of the present application; Figure 8 Another flowchart of an image processing method provided by an exemplary embodiment of the present application; Figure 9 A schematic structural diagram of an image processing device provided by an exemplary embodiment of the present application; Figure 10 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0014] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0015] In practical applications of machine vision imaging, high-quality imaging is crucial for subsequent visual inspection and analysis. However, in actual production, machine duplication is often required. This means that the same hardware (light source, lens, camera) is used to image the same product on the original machine. However, the images of the same product on different machines can be affected by factors such as the production environment, machine differences, and hardware aging, resulting in inconsistencies in texture, brightness, contrast, and color. This increases the complexity and false positive rate of subsequent inspection algorithms. A "machine" specifically refers to an independent, complete, and deployable visual imaging inspection unit, typically consisting of hardware (such as light sources, lenses, and cameras), mechanical structures (such as positioning structures like precision slides, rotary stages, and fixtures, as well as protective structures like dust covers, vibration-damping platforms, and temperature control boxes), and a control system (such as an industrial computer running parameter control software and image processing algorithms). Therefore, there is an urgent need for an image processing method that can ensure image consistency during "machine duplication." "Machine duplication" should be understood as imaging the same product on a new machine with the same hardware model as the original. In view of this, an embodiment of the present application provides an image processing method.

[0016] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes the following steps: Step 101: Acquire a first image group obtained by capturing images of a target object using a first machine based on first set imaging parameters, where the first machine includes hardware equipment for image capture.

[0017] Step 102: Acquire multiple second image groups based on multiple second set imaging parameters by capturing images of the target object using a second machine. The second machine includes the same hardware as the first machine. The multiple second set imaging parameters are obtained by adjusting the first set imaging parameters by setting adjustment ranges. The second set imaging parameters correspond one-to-one to the second image groups.

[0018] Step 103: Based on the set marked area of ​​each first image in the first image group, the marked corresponding area of ​​each second image in each second image group is scored for consistency to obtain a local consistency score, where the marked corresponding area corresponds to the set marked area.

[0019] Step 104 : Based on each first image in the first image group, perform a consistency score on each second image in each second image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image.

[0020] Step 105 : Determine a target consistency score according to the local consistency score and the global consistency score, and determine a second image group that meets the set consistency score requirement and second set imaging parameters corresponding to the second image group based on the target consistency score.

[0021] In actual applications, workers can determine in advance how to select the hardware equipment such as the light source, lens, camera, etc. in the first machine, as well as the first set imaging parameters Hi (such as light source brightness, angle, lens focal length, aperture, camera exposure time, gain, frame rate, resolution, etc.) based on the workpiece characteristics of the target object P1 (i.e., the workpiece to be inspected, such as a curved workpiece, polished glass, aluminum alloy housing, etc.).

[0022] For example, assuming that the target object P1 is a mobile phone's glossy glass screen, when scratch detection is required on the glossy glass, it is necessary to comprehensively consider the impact of factors such as "glossy surface", "micron-level scratches", and "glass reflection" on the imaging process. To this end, you can choose: 1. Camera parameters: "Model: SCI-XM1200-LMC4K-3, Quantity: 3 units, Resolution: 8192, Pixel size: 7μm, Color mode: Black and White, Exposure time: 33μs, Pixel resolution: 9.766μm / pixel, Gamma value: 1, Gain: 1".

[0023] 2. Lens related parameters: "Model: SCI-AVGP116 / 4.8-0.75X, focal length: 116mm, aperture: F4.8, field of view: 80mm, working distance: 231±5mm, depth of field: 1.8mm".

[0024] 3. Light source related parameters: "Light source 1, model: SCI-LSXSC440-W, linear coaxial light, brightness: 230, working distance: 100mm; Light source 2, model: SCI-LSX5440U-W, line light, brightness: 50, working distance: 90mm".

[0025] The first machine is also called the reference machine M1, which may be the machine with the best imaging effect and stable hardware status in the production line.

[0026] In a specific implementation, the following description will continue using the target object P1 as a smooth glass screen of a mobile phone as an example. After using the reference machine M1 to capture an image of the target object P1 based on the first set imaging parameters Hi, a first image group consisting of multiple first images (e.g., 8, 10, etc., not limited here) can be obtained. Each first image has a set marked area. The set marked area can be a region of interest (ROI), which can be drawn in the first image by a staff member using any shape such as a box, circle, ellipse, or irregular polygon. For example, the ROI can be the display area, camera aperture area, or edge area of ​​the mobile phone screen.

[0027] Afterwards, a second machine M2 is used to capture images of the target object P1 based on a plurality of second set imaging parameters Hj to obtain a plurality of second image groups, wherein the second machine M2 is a new machine having the same hardware equipment as the reference machine M1.

[0028] In actual application, based on the first set imaging parameter Hi, the first set imaging parameter Hi is adjusted several times in small increments to obtain a plurality of second set imaging parameters Hj. For ease of understanding, for example, assuming that the first set imaging parameter Hi includes "the brightness of the light source is 230, and the working distance is 100 mm", then the second set imaging parameter Hj corresponding to the first second image group may include "the brightness of the light source is 220, and the working distance is 95 mm" (that is, the multiple second images in the first second image group are imaged under "the brightness of the light source is 220, and the working distance is 95 mm"), the second set imaging parameter Hj corresponding to the second second image group may include "the brightness of the light source is 210, and the working distance is 90 mm" (that is, the multiple second images in the second second image group are imaged under "the brightness of the light source is 210, and the working distance is 90 mm"), the second set imaging parameter Hj corresponding to the third second image group may include "the brightness of the light source is 240, and the working distance is 110 mm" (that is, the multiple second images in the third second image group are imaged under "the brightness of the light source is 240, and the working distance is 110 mm"), and so on. I will not list them one by one here.

[0029] After obtaining the first image group and multiple second image groups, a consistency score can be performed on the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group to obtain a local consistency score, and the marked corresponding area corresponds to the set marked area; based on each first image in the first image group, a consistency score is performed on each second image in each second image group to obtain a global consistency score, and the target consistency score is determined based on the local consistency score and the global consistency score. The detailed process of the consistency scoring can be referred to the subsequent embodiments and will not be introduced in detail here.

[0030] After obtaining the target consistency score, the second image group whose consistency score meets the set requirements and the second set imaging parameters corresponding to the second image group are selected as the preferred imaging data for the second machine. After that, subsequent production inspection tasks can be carried out based on the second set imaging parameters.

[0031] For example, how to determine the consistency score that meets the set requirements: taking the first image group and any second target image group in multiple second image groups as an example, for each second image bjk in the second target image group B_j, respectively, calculate the consistency score with each image gi in the first image group G, and obtain the score matrix ,in, Indicates that this is a matrix with N rows and M columns. Then, the score matrix Take the maximum value by row and the mean by column to get the final consistency score. The specific formula is as follows: S_j=mean( max(S_i,:)),( S= mean(max(S,axis=1)) ) Where i represents the image index in the first image group G, i=1, 2...N; j represents the index of the second image group, representing the j-th second image group; max(S, axis=1): takes the maximum value of each row of the matrix S to obtain a vector, mean(...): takes the average of the above vectors to obtain S_j.

[0032] For ease of understanding, let's take an example: Assume that the first image group contains three first images, namely g1, g2 and g3, and the second target image group (such as the second image group of group 2) contains five second images, namely b21, b22, b23, b24, and b25. Then the score matrix formed by the first image group and the second target image group is as follows: b21 b22 b23 b24 b25 g1 0.92 0.85 0.88 0.90 0.82 g2 0.78 0.95 0.80 0.83 0.91 g3 0.88 0.80 0.93 0.87 0.84 In this implementation, we take the maximum value per row. The highest consistency score in the row containing first image g1 is 0.92, indicating that second image b21 is most similar to first image g1. Similarly, we can determine that the highest consistency score in the row containing first image g2 is 0.95, indicating that second image b22 is most similar to first image g2. The highest consistency score in the row containing first image g3 is 0.93, indicating that second image b23 is most similar to first image g3.

[0033] Then, take the average of the three highest scores 0.92, 0.95, and 0.93, that is: (0.92+0.95+0.93) / 3≈0.93 The 0.93 score is the consistency score between the second target image group and the first image group. By comparing all second image groups with the first image group in this manner, multiple consistency scores can be obtained. The highest score among these multiple consistency scores is determined as the consistency score that meets the set requirements. Next, the second image group whose consistency score meets the set requirements is determined, along with the second set imaging parameters corresponding to the second image group.

[0034] Through the above-mentioned method of "generating a score matrix, taking the maximum value by row and the mean by column in the score matrix, and then obtaining the final consistency score", consistency scores can be performed on multiple second image groups and the first image group to ensure that the final consistency score meets the set requirements of the second image group, and the second set imaging parameters corresponding to the second image group can better meet the subsequent detection requirements.

[0035] It should be noted that in actual applications, the number of machines is not limited. In other words, in addition to the first and second machines mentioned above, there may also be a third machine, a fourth machine, and so on. This is not limited here.

[0036] Based on the above, the image processing method provided in the embodiments of the present application achieves the acquisition of all images requiring consistency scoring by acquiring a first image group obtained by capturing images of a target object using a first machine based on first set imaging parameters, and multiple second image groups obtained by capturing images of the target object using a second machine based on multiple second set imaging parameters. Local consistency scores are obtained by performing consistency scoring on the marked areas corresponding to the marks of each second image in each second image group based on the set marked areas of each first image in the first image group. Global consistency scores are obtained by performing consistency scoring on each second image in each second image group based on each first image in the first image group. This ensures that the resulting target consistency score comprehensively considers both local and global factors, overcomes the impact of hardware differences and environmental fluctuations on the score, and ensures the accuracy of the target consistency score. Subsequently, based on the more accurate target consistency score, a second image group that meets the set consistency score requirements and the second set imaging parameters corresponding to the second image group are determined, ensuring image consistency during the "machine replication" process and laying the foundation for the accurate completion of subsequent inspection tasks such as defect detection and size detection of the target object.

[0037] In practical applications, in order to smoothly and accurately complete the acquisition of local consistency scores, before performing consistency scoring on the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group, the marked corresponding area of ​​each second image in each second image group can be determined based on the set marked area of ​​each first image in the first image group.

[0038] Figure 2 The flowchart for determining the marked corresponding area of ​​each second image in each second image group provided in the embodiment of the present application is as follows: Figure 2 As shown, based on the set marked area of ​​each first image in the first image group, determining the marked corresponding area of ​​each second image in each second image group includes the following steps: Step 201: Input a first target image and a second target image into a feature detection model to obtain feature points and descriptors corresponding to the first target image, as well as feature points and descriptors corresponding to the second target image, wherein the first target image is one of multiple first images in a first image group, and the second target image is one of multiple second images in a second image group.

[0039] Step 202: Filter out the feature points and descriptors of the set marked area of ​​the first target image from the feature points and descriptors corresponding to the first target image.

[0040] Step 203 : Perform feature matching based on the feature points and descriptors corresponding to the second target image and the feature points and descriptors of the set marked area of ​​the first target image to determine a first mapping relationship between the second target image and the set marked area of ​​the first target image.

[0041] Step 204: Determine a marked area corresponding to the set marked area in the second target image based on the first mapping relationship.

[0042] In practical applications, for ease of understanding, the following Figure 3 Let's take an example: When implementing it specifically, Figure 3 As shown, the first target image gi and the second target image bjk are respectively input into a pre-trained feature detection model to obtain first feature data and second feature data output by the feature detection model. The first feature data includes the feature points and descriptors of the first target image gi, from which feature point descriptors for a defined marked region (ROI) are selected (selection can be performed by setting the coordinates of the four corners corresponding to the marked region (ROI). The second feature data includes the feature points and descriptors for the entire second target image. Feature points can be dense feature points obtained by the dense feature matching algorithm in the feature detection model and represented in coordinate form. Descriptors can generally have dimensions such as 64d, 128d, or 256d, and are used to characterize the texture information surrounding the feature points. It should be noted that dense feature points can provide rich contextual information and are more beneficial for aligning regions with weak texture or repetitive structures. The dimension of the descriptor is preferably 64d (dense detection has a large number of feature points, and low dimensionality facilitates faster matching).

[0043] The feature detection model is a deep learning model that follows the following requirements during training: (1) Based on the dense feature point detection structure, the self-supervised method is used to train industrial data; among them, industrial data has the following characteristics: the contrast and richness are not as good as natural image data; there are many repetitive structural areas.

[0044] (2) Feature invariance learning: Explicitly introduce illumination changes, perspective changes, scale and rotation changes during training, so that the features have illumination, perspective, scale and rotation invariance, which is more conducive to target positioning and target alignment tasks in industrial data; (3) Speed: If the model output dimension is set to 64d, the dimension of the descriptor is 64d.

[0045] Afterwards, feature matching is performed between the feature points of the set marked area and the feature points of the second target image using a mutual nearest neighbor method. Similarity algorithms such as Euclidean distance and cosine similarity can be used to calculate the similarity between the feature points of the set marked area and the feature points of the second target image, resulting in multiple sets of feature point pairs with similarities greater than a set threshold. Randomly select N sets of feature point pairs (N can be 4, 5, 6, etc., and can be set according to actual needs) from the multiple feature point pairs. Assuming N=4, calculate the homography matrix M (3×3 transformation matrix), count the number of inliers that satisfy the homography matrix M, and repeat this process 1000 times. Select the homography matrix M with the largest number of inliers (i.e., the first mapping relationship between the set marked area and the second target image). A specific example of the homography matrix M is as follows (the specific values ​​of A1, B1, C1, D1, E1, and F1 can be determined according to actual conditions): M= Then, based on the homography matrix M, the vertex coordinates of the marked region ROI are mapped to the vertex coordinates of the marked region ROI in the second target image. Specifically, the vertex coordinates of sROI = M·vertex coordinates of ROI. For example, assuming that the four vertex coordinates of the marked region ROI are [1274, 531], [4104, 531], [4104, 3348], and [1274, 3348], respectively, after M mapping, the four vertex coordinates of the marked region sROI can be [1814, 157], [4514, 1012], [3666, 3648], and [966, 2845].

[0046] The position of the marked corresponding region sROI in the second target image can be accurately determined by using the vertex coordinates of the marked corresponding region sROI.

[0047] By extracting the first feature data corresponding to the set marked area in the first target image and the second feature data corresponding to the second target image, the mapping relationship between the set marked area and the second target image can be determined, so as to determine the marked corresponding area sROI in the second target image according to the mapping relationship, thereby laying the foundation for subsequent local consistency scoring and global consistency scoring, and thus obtaining a more accurate consistency score.

[0048] After determining the set marked region ROI and the marked corresponding region sROI, based on the set marked region and the marked corresponding region, respectively, a set marked region image block is cropped in the first target image, and a marked corresponding region image block is cropped in the second target image; a second mapping relationship between the set marked region image block and the marked corresponding region image block is determined; based on the second mapping relationship, the marked corresponding region image block is geometrically aligned to obtain an aligned marked corresponding region image block; feature extraction is performed on the marked corresponding region image block and the aligned marked corresponding region image block, and feature difference calculation is performed based on the result of the feature extraction; and a local consistency score is determined according to the result of the feature difference calculation.

[0049] When implementing it specifically, Figure 4 As shown, feature extraction can be performed on the set marked area image block Patch_g and the marked corresponding area image block Patch_s based on the method in the above embodiment, and a second mapping relationship M'' between the set marked area image block Patch_g and the marked corresponding area image block Patch_s can be determined, or a second mapping relationship M'' can be obtained based on the first mapping relationship M, and geometric alignment processing is performed on the marked corresponding area image block Patch_s based on the second mapping relationship M''. Specifically, the central rotation matrix of Patch_s is calculated through the rotation and scaling amount of the first mapping relationship M (the 2×2 submatrix in the upper left corner) and the size of Patch_s, that is, the second mapping relationship M'': =The inverse of the upper left 2×2 submatrix of M = [[A2,B2],[D2,E2]]; Patch_s has a height and width of (h, w). In the second image, the upper left corner is marked as (x0, y0), and the lower right corner is marked as (x1, y1). Then, T1=[[1,0,-(w-1) / 2], [0,1,-(h-1))2], [0,0,1]] T2=[[A2,B2,0], [D2,E2,0], [0,0,1]] T3=[[1,0,(w-1) / 2], [0,1,(h-1))2], [0,0,1]] M''=T1·T2·T3. T1, T2, and T3 are transformation matrices. T1 moves the image coordinate system origin from the upper left corner to the center of the image, T2 performs rotation and scaling correction in the central coordinate system, and T3 moves the coordinate system origin from the center back to the upper left corner.

[0050] The coordinates of sROI in Patch_s are sROI': sROI' = sROI - (x0, y0). Therefore, after Patch_s and sROI' undergo the M'' transformation, we obtain: Patch_s'' = M''·Patch_s, sROI'' = M''·sROI'. Finally, sROI'' crops Patch_s'' to obtain Patch_s'. At this point, Patch_s' and Patch_g are of the same size and spatially aligned. By performing geometric alignment, the geometric differences between the two image patches can be eliminated through spatial transformation, making them completely match in shape, size, and orientation, eliminating non-content differences and ensuring that the subsequent consistency score only compares intrinsic properties such as texture and brightness.

[0051] For ease of understanding, Figure 5 The specific data change process of Patch_s' and Patch_g alignment is shown in FIG. Figure 5 In the example, the height and width of the first target image are (3648, 5472), and the coordinates of the four corners of the marked region ROI are set to [[1274, 531], [4104, 531], [4104, 3348], [1274, 3348]]. The height and width of the second target image are (3648, 5472), and the coordinates of the four corners of the marked region are sROI: [[1814, 157], [4514, 1012], [3666, 3648], [966, 2845]]. The height and width of the marked area image block Patch_g are set to (2817, 2830), the height and width of the marked corresponding area image block Patch_s are set to (3491, 3548), the coordinates of its upper left corner are (966, 157), and the coordinates of its lower right corner are (4514, 3648). After the central rotation matrix M'' is transformed, the height and width of the marked corresponding area image block Patch_s'' are (3491, 3548). After sROI'' is used to crop Patch_s'', Patch_s' is obtained. The height and width of Patch_s' are (2817, 2830), which are consistent with the height and width of the set marked area image block Patch_g.

[0052] After geometric alignment, Figure 5 and Figure 6As shown in the figure, the set marked area image block Patch_g and the marked corresponding area image block Patch_s' are respectively input into the Convolutional Neural Network (CNN), and the feature vectors fg and fs' output by the Convolutional Neural Network CNN are obtained to extract the deep abstract features such as texture, brightness, and contrast in the set marked area image block Patch_g and the marked corresponding area image block Patch_s', while ignoring irrelevant noise (such as slight moiré).

[0053] Afterwards, the feature difference between the eigenvectors fg and fs' is calculated, i.e., |fg−fs'|. By calculating the feature difference, the degree of deviation between the two image blocks in the feature space can be quantified. The larger the value of the feature difference, the lower the consistency of the two image blocks. Conversely, the smaller the value of the feature difference, the higher the consistency of the two image blocks.

[0054] By inputting the feature difference |fg−fs'| into a multilayer perceptron (MLP), we obtain the local consistency score P_S output by the MLP. This local consistency score P_S reflects the consistency of the local region in terms of texture, brightness, contrast, etc. This method lays the foundation for obtaining a more accurate target consistency score.

[0055] Afterwards, a third mapping relationship between the first target image and the second target image is determined based on the feature points and descriptors corresponding to the first target image, as well as the feature points and descriptors corresponding to the second target image; based on the third mapping relationship, the second target image is geometrically aligned to obtain an aligned second target image; features are extracted from the first target image and the aligned second target image, and feature differences are calculated based on the results of the feature extraction; and a global consistency score is determined based on the results of the feature difference calculation.

[0056] Specifically, if Figure 7 As shown, based on the method adopted in the above-mentioned process of calculating the local consistency score P_S, feature matching is performed on the first target image gi and the second target image bjk, and a third mapping relationship (i.e., the homography matrix M') between the two is calculated. Based on the homography matrix M', the second target image bjk is geometrically aligned to obtain bjk'.

[0057] The first target image gi and the aligned second target image bjk' are input into the image quality assessment network composed of the above-mentioned convolutional neural network CNN and multi-layer perceptron MLP (the specific implementation process can be found in the above embodiment and will not be repeated here). The global consistency score G_S can be obtained. The global consistency score G_S reflects the consistency of the first target image gi and the aligned second target image bjk' in global features.

[0058] After determining the local consistency score and the global consistency score, a mean operation can be performed on the local consistency score and the global consistency score, and the result of the mean operation can be used as the consistency score for each second image in the second image group, that is, (P_S + G_S) / 2. By performing the mean operation on the local consistency score and the global consistency score, a final consistency score can be determined while comprehensively considering both the local and global consistency scores, thereby ensuring the accuracy of the consistency score.

[0059] Figure 8 Another flow chart of an image processing method provided in an embodiment of the present application is as follows: Figure 8 As shown, the method includes the following steps: Step 801: Acquire a first image group obtained by capturing images of a target object using a first machine based on first set imaging parameters, where the first machine includes hardware equipment for image capture.

[0060] Step 802: Acquire multiple second image groups based on multiple second set imaging parameters, obtained by respectively capturing images of the target object using a second machine. The second machine includes the same hardware equipment as the first machine. The multiple second set imaging parameters are respectively obtained by adjusting the first set imaging parameters by setting an adjustment range. The second set imaging parameters correspond one-to-one to the second image groups.

[0061] Step 803: Based on the set marked area of ​​each first image in the first image group, the marked corresponding area of ​​each second image in each second image group is scored for consistency to obtain a local consistency score, where the marked corresponding area corresponds to the set marked area.

[0062] Step 804 : Based on each first image in the first image group, perform a consistency score on each second image in each second image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image.

[0063] Step 805 : Determine a target consistency score according to the local consistency score and the global consistency score, and determine a second image group that meets the set consistency score requirement and second set imaging parameters corresponding to the second image group based on the target consistency score.

[0064] Step 806: If it is determined that the image quality of the second image group does not meet the set image quality detection requirement, adjust the second set imaging parameters.

[0065] The specific execution process of steps 801 to 805 can be found in the above embodiment and will not be described again here.

[0066] For step 806, in actual application, after step 805 determines the second image group and the second set imaging parameters corresponding to the second image group, the second image group and the second set imaging parameters corresponding to the second image group can be submitted to the staff for manual confirmation. If it is manually confirmed that the image quality of the second image in the second image group does not meet the set image quality detection requirements, the second set imaging parameters can be fine-tuned or iteratively optimized until the image quality of the second image meets the set image quality detection requirements.

[0067] By adjusting the second set imaging parameters when it is determined that the image quality of the second image in the second image group does not meet the set image quality detection requirements, a foundation is laid for meeting subsequent high-precision production requirements.

[0068] Based on the above, the image processing method provided in the embodiments of the present application has at least the following beneficial effects: 1. Improve efficiency and reduce costs: Through the automated image consistency evaluation algorithm, the optimal imaging parameters can be quickly screened, reducing the time and labor costs of manually debugging light source, lens, and camera parameters during machine replication.

[0069] 2. Ensure imaging consistency: Combine local and global image evaluation to overcome hardware differences and environmental fluctuations, and ensure that the new machine image is highly consistent with the reference image group in terms of texture, brightness, etc.

[0070] 3. Enhanced detection robustness: Highly consistent images reduce the risk of misjudgment caused by image differences in the detection algorithm, improving detection accuracy. This is particularly suitable for tasks that are sensitive to local features.

[0071] 4. Flexible and universal: The algorithm is applicable to a variety of artifacts and scenarios. The modular design supports algorithm replacement and optimization and has good scalability.

[0072] 5. Promote intelligent production: Automated parameter optimization is in line with the trend of intelligent manufacturing, provides high-quality image data, and supports deep learning and big data analysis.

[0073] 6. Easy to optimize and confirm: Output quantitative scores and optimal parameters to facilitate manual confirmation and iterative optimization to meet high-precision production needs.

[0074] Figure 9A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Figure 9 As shown, the device includes: an acquisition module 11, a scoring module 12 and a determination module 13.

[0075] The acquisition module 11 is used to acquire a first image group obtained by acquiring an image of a target object using a first machine based on first set imaging parameters, where the first machine includes hardware equipment for image acquisition; and to acquire a plurality of second image groups obtained by acquiring images of the target object using a second machine based on a plurality of second set imaging parameters, where the second machine includes the same hardware equipment as the first machine, and the plurality of second set imaging parameters are respectively obtained by adjusting the first set imaging parameters by setting an adjustment range, and the second set imaging parameters correspond one-to-one to the second image groups.

[0076] The scoring module 12 is configured to perform a consistency score on the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group to obtain a local consistency score, where the marked corresponding area corresponds to the set marked area; and to perform a consistency score on each second image in each second image group based on each first image in the first image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image.

[0077] The determination module 13 is configured to determine a target consistency score according to the local consistency score and the global consistency score, so as to determine a second image group that meets the set consistency score requirements and second set imaging parameters corresponding to the second image group based on the target consistency score.

[0078] Optionally, the determination module 13 is also used to: determine the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group; and input the first target image and the second target image into the feature detection model to obtain feature points and descriptors corresponding to the first target image, and feature points and descriptors corresponding to the second target image, wherein the first target image is one of the multiple first images in the first image group, and the second target image is one of the multiple second images in the second image group; among the feature points and descriptors corresponding to the first target image, filter out the feature points and descriptors of the set marked area of ​​the first target image; perform feature matching based on the feature points and descriptors corresponding to the second target image, and the feature points and descriptors of the set marked area of ​​the first target image to determine a first mapping relationship between the second target image and the set marked area of ​​the first target image; and determine the marked corresponding area in the second target image corresponding to the set marked area based on the first mapping relationship.

[0079] Optionally, the scoring module 12 is specifically configured to: based on the set marked area and the marked corresponding area, crop a set marked area image block in the first target image and crop a marked corresponding area image block in the second target image; determine a second mapping relationship between the set marked area image block and the marked corresponding area image block; based on the second mapping relationship, perform geometric alignment processing on the marked corresponding area image block to obtain an aligned marked corresponding area image block; perform feature extraction on the marked corresponding area image block and the aligned marked corresponding area image block, and perform feature difference calculation based on the result of the feature extraction; determine a local consistency score based on the result of the feature difference calculation. And, based on the feature points and descriptors corresponding to the first target image and the feature points and descriptors corresponding to the second target image, determine a third mapping relationship between the first target image and the second target image; based on the third mapping relationship, perform geometric alignment processing on the second target image to obtain an aligned second target image; perform feature extraction on the first target image and the aligned second target image, and perform feature difference calculation based on the result of the feature extraction; determine a global consistency score based on the result of the feature difference calculation.

[0080] Optionally, the determining module 13 is further configured to perform a mean operation on the local consistency score and the global consistency score, and use the result of the mean operation as a target consistency score.

[0081] Optionally, the apparatus further includes: an adjustment module, configured to adjust the second set imaging parameters if it is determined that the image quality of the second image group does not meet a set image quality detection requirement.

[0082] Figure 9 The device shown can execute the steps of the image processing method in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment and will not be repeated here.

[0083] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 As shown, in practice, the electronic device includes: a memory 21 and a processor 22.

[0084] The memory 21 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, data structures, contact data, phone book data, messages, images, videos, etc.

[0085] The processor 22 is coupled to the memory 21 and is configured to execute the computer program in the memory 21 to implement the file detection method provided in the aforementioned embodiment.

[0086] Further, if Figure 10 As shown, the electronic device also includes: a communication component 23, a display 24, a power component 25, an audio component 26 and other components. Figure 10 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 10 The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone or an IOT device, or a server device such as a conventional server, a cloud server or a server array.

[0087] The above-mentioned memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0088] The communication component is configured to facilitate wired or wireless communication between the device in which the communication component resides and other devices. The device in which the communication component resides can access a wireless network based on a communication standard, such as a 2G, 3G, 4G / LTE, 5G, or other mobile communication network, or a combination thereof. In an exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0089] The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, it may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can detect not only the boundaries of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.

[0090] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0091] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC). When the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in the memory or sent via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0092] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement each step in the above-mentioned method embodiment. The computer-readable storage medium includes volatile or non-volatile storage, or a combination thereof, and may be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape, disk storage or other magnetic storage devices, or any other non-transmission medium.

[0093] Accordingly, an embodiment of the present application further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the processor is enabled to implement the steps in the above-mentioned method embodiment. It should be understood that each process or a combination of multiple processes in the above-mentioned method flow can be implemented by a computer program or instruction. In addition, these computer programs or instructions can be applied to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device, so that the processor of the general-purpose computer, the special-purpose computer, the embedded processor or other programmable data processing device can be implemented as a device for implementing the corresponding functions in the above-mentioned method embodiment.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: include: Acquiring a first image group obtained by capturing an image of a target object using a first machine based on first set imaging parameters, wherein the first machine includes a hardware device for performing image capture; Acquiring a plurality of second image groups obtained by respectively capturing images of the target object using a second machine based on a plurality of second set imaging parameters, wherein the second machine comprises the same hardware device as the first machine, the plurality of second set imaging parameters being obtained by adjusting the first set imaging parameters by setting adjustment ranges, and the second set imaging parameters and the second image groups having a one-to-one correspondence; Based on the set marked area of ​​each first image in the first image group, performing a consistency score on the marked corresponding area of ​​each second image in each second image group to obtain a local consistency score, the marked corresponding area corresponding to the set marked area; performing a consistency score on each second image in each second image group based on each first image in the first image group to obtain a global consistency score, where the consistency score is used to reflect the consistency between the second image and the first image; A target consistency score is determined according to the local consistency score and the global consistency score, so as to determine a second image group that meets a set consistency score requirement and second set imaging parameters corresponding to the second image group based on the target consistency score.

2. The method according to claim 1, characterized in that Before performing consistency scoring on the marked corresponding area of ​​each second image in each second image group based on the set marked area of ​​each first image in the first image group to obtain the local consistency score, the method further includes: Based on the set marked area of ​​each first image in the first image group, the marked corresponding area of ​​each second image in each second image group is determined.

3. The method according to claim 2, characterized in that The determining, based on the set marked area of ​​each first image in the first image group, the marked corresponding area of ​​each second image in each second image group includes: Inputting a first target image and a second target image into a feature detection model to obtain feature points and descriptors corresponding to the first target image, and feature points and descriptors corresponding to the second target image, wherein the first target image is one of the multiple first images in the first image group, and the second target image is one of the multiple second images in the second image group; Filtering the feature points and descriptors corresponding to the first target image to select the feature points and descriptors of the set marked area of ​​the first target image; performing feature matching based on feature points and descriptors corresponding to the second target image and feature points and descriptors of the set marked area of ​​the first target image to determine a first mapping relationship between the second target image and the set marked area of ​​the first target image; Based on the first mapping relationship, a mark corresponding area in the second target image corresponding to the set mark area is determined.

4. The method according to claim 3, characterized in that Based on the set marked area of ​​each first image in the first image group, performing consistency scoring on the marked corresponding area of ​​each second image in each second image group to obtain a local consistency score, including: Based on the set marked area and the marked corresponding area, cropping an image block of the set marked area in the first target image and cropping an image block of the marked corresponding area in the second target image; Determining a second mapping relationship between the image block in the set marked area and the image block in the marked corresponding area; Based on the second mapping relationship, geometric alignment processing is performed on the image blocks of the marked corresponding areas to obtain aligned image blocks of the marked corresponding areas; performing feature extraction on the image block of the marked corresponding region and the aligned image block of the marked corresponding region, and performing feature difference calculation based on the result of the feature extraction; A local consistency score is determined based on the result of the feature difference calculation.

5. The method according to claim 3, characterized in that The step of performing a consistency score on each second image in each second image group based on each first image in the first image group to obtain a global consistency score includes: determining a third mapping relationship between the first target image and the second target image according to the feature points and descriptors corresponding to the first target image and the feature points and descriptors corresponding to the second target image; Based on the third mapping relationship, performing geometric alignment processing on the second target image to obtain an aligned second target image; performing feature extraction on the first target image and the aligned second target image, and performing feature difference calculation based on results of the feature extraction; A global consistency score is determined based on the result of the feature difference calculation.

6. The method according to claim 1, wherein Determining a target consistency score according to the local consistency score and the global consistency score includes: A mean operation is performed on the local consistency score and the global consistency score, so as to use a result of the mean operation as a target consistency score.

7. The method according to any one of claims 1 to 6, characterized in that Also includes: If it is determined that the image quality of the second image group does not meet the set image quality detection requirement, the second set imaging parameters are adjusted.

8. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image processing method according to any one of claims 1 to 7.

9. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the image processing method according to any one of claims 1 to 7.

10. A computer program product, characterized in that include: A computer program, when executed by a processor of an electronic device, causes the processor to execute the image processing method according to any one of claims 1 to 7.

Images