Image processing method, image processing device and detection equipment

Abstract

The present invention provides an image processing method, an image processing device and a detection device. Based on the present invention, a detection mask image can be created using a detection area determined by regional calibration information of ROI and ROS. Since the detection area is defined in the non-overlapping area of ​​ROI and ROS, the detection mask image created using the detection area can have the effect of using ROS to partially shield the ROI. Thus, the detection mask image whose effective area is constrained in the detection area is used to perform regional validity filtering on the target image, and only the original image value in the local area of ​​the target image that is enclosed in the ROI and not shielded by the ROS can be retained, thereby eliminating internal interference and redundant image processing in the ROS, which helps to improve the image processing recognition accuracy and image processing efficiency of the target image. Moreover, the regional calibration information of ROI and ROS can be used to perform posture compensation with reference to the reference features, so that the targeted positioning of the local shielding effect in the target image can be more accurately achieved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to an image processing method, an image processing device, and a detection device for defining an image detection area by using ROI (Region of interest) and ROS (Region of shield). Background Art

[0002] Image detection often focuses on a certain part of the image. How to provide effective support for local image detection has become a technical problem to be solved in the existing technology. Summary of the invention

[0003] In view of this, each embodiment of the present invention provides an image processing method, an image processing apparatus, and a detection device.

[0004] In one embodiment, an image processing method is provided, comprising:

[0005] Obtaining regional calibration information of a region of interest ROI and regional calibration information of a shielding region ROS located in the ROI;

[0006] Determine a detection area according to the area calibration information, wherein the detection area is a reserved area after the overlapping area with the ROS in the ROI is removed;

[0007] Creating a detection mask image whose effective area is constrained within the detection area;

[0008] The detection mask image is used to perform region validity filtering on the target image.

[0009] Optionally, before obtaining the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI, it further includes: detecting the boundary position conditions of the ROI and ROS calibrated in the sample image in a human-computer interaction manner, wherein the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size; and creating the regional calibration information of the ROI and ROS using the detected boundary position conditions.

[0010] Optionally, before obtaining the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI, it further includes: detecting a sample classification instruction input in a human-computer interaction manner, and associating a category identifier with the created regional calibration information of the ROI and ROS, wherein the sample classification instruction includes the category identifier of the feature object presented in the sample image; obtaining the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI includes: detecting a region selection instruction input in a human-computer interaction manner, and calling the region calibration information of the ROI and ROS associated with the category identifier in the region selection instruction, wherein the region selection instruction includes the category identifier of the feature object presented in the target image to be processed.

[0011] Optionally, determining the detection area according to the area calibration information includes: obtaining a detection area position set by performing a difference operation on a ROI position set represented by the area calibration information of the ROI and a ROS position set represented by the area calibration information of the ROS; creating a detection mask image in which the valid area is constrained within the detection area includes: loading a mask base plate image having the same size as the target image, and obtaining a detection mask image having the same size as the target image by setting the area in the mask base plate image that overlaps with the detection area position set to a valid value; performing area validity filtering on the target image using the detection mask image includes: obtaining a composite image by performing a logical AND operation on aligned pixels of the detection mask image and the target image, wherein the composite image retains the original pixel array of the target image within the detection area and is set to an invalid pixel array outside the detection area.

[0012] Optionally, when the region calibration information of ROI represents multiple ROI position sets, determining the detection area according to the region calibration information further includes: performing a union operation on the multiple ROI position sets before performing a difference operation on the ROI position set and the ROS position set; when the region calibration information of ROS represents multiple ROS position sets, determining the detection area according to the region calibration information further includes: performing a union operation on the multiple ROS position sets before performing a difference operation on the ROI position set and the ROS position set.

[0013] Optionally, before determining the detection area according to the area calibration information, it further includes: detecting a reference feature calibrated in the sample image in a human-computer interaction manner, wherein the reference feature is located outside the range of the ROI and has a fixed position relationship with the ROI; detecting the calibration pose of the reference feature in the sample image; detecting the posture deviation of the current posture of the reference feature in the target image relative to the calibration pose in the sample image.

[0014] Optionally, using the posture deviation to perform posture correction on the regional calibration information of ROI and ROS includes: using the coordinate deviation in the posture deviation to perform coordinate translation on the regional calibration information of ROI and ROS; and / or, taking the position coordinates of the reference feature in the target image as the center, using the angle deviation in the posture deviation to perform deflection angle compensation on the regional calibration information of ROI and ROS.

[0015] In another embodiment, an image processing apparatus is provided, comprising:

[0016] A calibration loading module is used to obtain the regional calibration information of the region of interest ROI and the regional calibration information of the shielding area ROS located in the ROI;

[0017] A region definition module, used to determine a detection region according to the region calibration information, wherein the detection region is a reserved region after the overlapping region with the ROS in the ROI is removed;

[0018] A mask creation module, used for creating a detection mask image whose effective area is constrained within the detection area;

[0019] The image filtering module is used to perform regional validity filtering on the target image using the detection mask image.

[0020] Optionally, it further includes: a calibration creation module, which is used to detect the boundary position conditions of the ROI and ROS calibrated in the sample image in a human-computer interaction manner, and use the detected boundary position conditions to create regional calibration information of the ROI and ROS, wherein the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size.

[0021] Optionally, the calibration creation module is further used to detect sample classification instructions input in a human-computer interaction manner, and to associate the category identifier with the created ROI and ROS region calibration information, wherein the sample classification instruction includes the category identifier of the feature object presented in the sample image; the calibration loading module is further used to detect region selection instructions input in a human-computer interaction manner, and to call the ROI and ROS region calibration information associated with the category identifier in the region selection instruction, wherein the region selection instruction includes the category identifier of the feature object presented in the target image to be processed.

[0022] Optionally, it further includes: a reference calibration module, used to detect the reference features calibrated in the sample image in a human-computer interaction manner, and detect the calibration pose of the reference features in the sample image; a posture compensation module, used to detect the posture deviation of the current posture of the reference features in the target image relative to the calibration pose in the sample image, and use the posture deviation to perform posture correction on the regional calibration information of ROI and ROS.

[0023] In another embodiment, a detection device is provided, comprising a processor, wherein the processor is used to execute the steps in the image processing method as described above.

[0024] In another embodiment, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to perform the steps in the image processing method described above.

[0025] Based on the above embodiments, a detection mask image can be created using the detection area determined by the regional calibration information of the ROI and the ROS. Since the detection area is defined in the non-overlapping area of ​​the ROI and the ROS, the detection mask image created using the detection area can have the effect of locally shielding the ROI using the ROS. Therefore, by using the detection mask image whose effective area is constrained in the detection area to perform regional validity filtering on the target image, only the original image value in the local area of ​​the target image that is enclosed in the ROI and not shielded by the ROS can be retained, thereby eliminating internal interference and redundant image processing in the ROS, which helps to improve the image processing recognition accuracy and image processing efficiency of the target image.

[0026] Moreover, as a further optimization, if the regional calibration information of ROI and ROS is compensated for posture with reference to the reference features, the local shielding effect of ROS on ROI can be more accurately targeted in the target image.

[0027] In addition, as a further optimization, if ROI and ROS can support customized settings for human-computer interaction, the shape of the detection area can be made universal and more in line with actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The following drawings are only used to illustrate and explain the present invention, and do not limit the scope of the present invention:

[0029] Figure 1 is a schematic diagram of an exemplary flow chart of an image processing method in one embodiment;

[0030] Figure 2 For Figure 1 Schematic diagram of examples of ROIs and ROS used in the image processing method shown;

[0031] Figure 3 For Figure 1 The image processing method shown is a schematic diagram of an extended process based on a human-computer interactive calibration method;

[0032] Figure 4 For Figure 1 The image processing method shown is a schematic diagram of an extended process based on position set operation;

[0033] Figure 5 For Figure 1 The image processing method shown is a schematic diagram of an extended process based on region quantity recognition;

[0034] Figure 6 For Figure 1 The image processing method shown is a schematic diagram of an extended process based on regional posture correction;

[0035] Figure 7 For Figure 6 A schematic diagram of an example of a calibration sample based on which the pose correction in the extended process is shown;

[0036] Figure 8 For example Figure 7 Schematic diagram of an example of a benchmark setting for a calibration sample shown;

[0037] Fig. 9 For the corresponding Figure 7 An example schematic diagram of an offset target sample of the calibration sample is shown;

[0038] Fig.10 For example Fig. 9 Schematic diagram of an example of posture correction of a shifted target sample;

[0039] Figures 11a to 11c Based on Figure 6 A schematic diagram of an example of image processing of the extended process shown;

[0040] Fig.12 is a schematic diagram of an exemplary structure of an image processing device in another embodiment;

[0041] Fig.13 is a schematic diagram of an extended structure of the image processing device shown in FIG11;

[0042] Fig.14 is another extended structural schematic diagram of the image processing device shown in FIG11;

[0043] Fig.15 It is a schematic structural diagram of a detection device in another embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0045] Figure 1 FIG. 1 is a schematic diagram of an exemplary flow chart of an image processing method in one embodiment. Figure 1 In one embodiment, an image processing method may include:

[0046] S110: Acquire the region calibration information of the ROI and the region calibration information of the ROS located in the ROI.

[0047] The ROI regional calibration information and ROS regional calibration information obtained in this step can represent the coordinate positions covered by the ROI and ROS in the image coordinate system, and the set of these coordinate positions can reflect the shape, area and posture of the ROI and ROS. That is, it can be considered that the ROI regional calibration information can represent the ROI position set, and the ROS regional calibration information can represent the ROS position set.

[0048] S120: determining a detection region according to the region calibration information, wherein the detection region is a reserved region after an overlapping region in the ROI with the ROS is removed.

[0049] The detection area determined in this step can also be considered as a set of coordinate positions in the image coordinate system.

[0050] S130: Creating a detection mask image in which the effective area is constrained within the detection area.

[0051] The detection mask image in this step can be considered as an attribute value image in which pixels are selectively set to valid values ​​or invalid values. The valid value can be set to 255, and each bit of its binary value 11111111 can be logically ANDed with the corresponding bits of other values ​​to keep the corresponding bits of other values ​​at their original values; the invalid value can be set to 0, and each bit of its binary value 00000000 can be logically ANDed with the corresponding bits of other values ​​to make the corresponding positions of other values ​​0 indicating invalidity. Accordingly, the valid area of ​​the detection mask image is constrained to the detection area, which can be understood as the detection mask image corresponding to the pixels in the detection area are valid values, and the pixels in other areas are invalid values.

[0052] S140: Performing regional validity filtering on the target image using the detection mask image.

[0053] The regional validity filtering implemented on the target image in this step can be considered as regional selection of the original image value of the target image by using the pixel attribute value of the detection mask image, that is, detecting the valid value points in the mask image can make the corresponding pixels in the target image retain the original value, and detecting the invalid value points in the mask image can assign the corresponding pixels in the target image to alternative values ​​that can be ignored during image detection, wherein assigning the corresponding pixels in the target image to ignored alternative values ​​can be considered as local de-value processing of the target image.

[0054] Figure 2 For Figure 1Schematic diagram of examples of ROIs and ROS used in the image processing methods shown. Figure 2 , based on Figure 1 The process shown can create a detection mask image using the detection area 200 determined by the area calibration information of the ROI 210 and the ROS 220. Since the detection area 200 is defined in the non-overlapping area of ​​the ROI 210 and the ROS 220, the detection mask image created using the detection area 200 can have the effect of partially shielding the ROI 210 using the ROS 220.

[0055] Therefore, by using the detection mask image whose effective area is constrained in the detection area to perform regional validity filtering on the target image, only the original image value in the local area of ​​the target image that is enclosed in the ROI and not shielded by the ROS can be retained, thereby eliminating internal interference and redundant image processing in the ROS, which helps to improve the image processing recognition accuracy and image processing efficiency of the target image.

[0056] When the above process is specifically implemented, the ROI and ROS can be calibrated manually, that is, calibrated in the sample image by using a virtual brush tool.

[0057] Figure 3 For Figure 1 The image processing method shown is based on the extended process diagram of the human-computer interactive calibration method. Figure 3 ,like Figure 1 The image processing method shown can be further extended to include the following steps:

[0058] S310: Detecting boundary position conditions of ROI and ROS calibrated in a sample image in a human-computer interaction manner, wherein the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size.

[0059] The boundary position conditions detected in this step may be discrete boundary features such as edge points.

[0060] S320: Create region calibration information of ROI and ROS using the detected boundary position conditions.

[0061] In the case where discrete boundary features such as edge points are detected in S310, the discrete boundary features may be fitted into closed polygons with the boundary points as vertices in this step, and the region calibration information may be created using the fitted closed polygons.

[0062] S330: Acquire the region calibration information of the ROI and the region calibration information of the ROS located in the ROI.

[0063] S340: determining a detection region according to the region calibration information, wherein the detection region is a reserved region after an overlapping region in the ROI with the ROS is removed.

[0064] S350: Create a detection mask image whose effective area is constrained within the detection area.

[0065] S360: Performing regional validity filtering on the target image using the detection mask image.

[0066] S330 to S360 in the above process can be considered as Figure 1 The principles of S110 to S140 shown in FIG. 1 are basically the same.

[0067] In practical applications, the above process can be performed multiple times for different sample images. The sample images mentioned here can be specifically understood as sample images presenting different categories of feature objects (such as license plate characters, product model identifiers, panel switch buttons, etc.). Therefore, by performing the above process multiple times, the regional calibration information of ROI and ROS applicable to feature objects of different categories can be obtained.

[0068] In order to distinguish and manage the regional calibration information of ROI and ROS applicable to different categories of feature objects, after each execution of the above process, the sample classification instruction input in a human-computer interaction manner can be further detected, and the category identifier can be associated with the created ROI and ROS regional calibration information, wherein the sample classification instruction includes the category identifier of the feature object presented in the sample image. In specific implementation, the sample classification instruction input in a human-computer interaction manner can be generated by selecting the corresponding option in the category entry menu of the human-computer interaction interface for calibrating ROI and ROS, indicating that the sample classification instruction generated when a category option is selected can automatically carry the category identifier of the category.

[0069] Accordingly, if a category identification management mechanism is introduced, then each time Figure 1 In the process shown, a category identification selection mechanism can also be introduced, that is, S110 can detect the region selection instruction input by human-computer interaction, and call the region calibration information of ROI and ROS associated with the category identification in the region selection instruction, wherein the region selection instruction contains the category identification of the feature object presented in the target image to be processed. In specific implementation, the region selection instruction input by human-computer interaction can be generated by selecting the corresponding option in the category entry menu of the human-computer interaction interface for detection configuration, indicating that the region selection instruction generated when the option of a category is selected can automatically carry the category identification of the category.

[0070] Figure 4 For Figure 1The image processing method shown is based on the extended flow diagram of position set operation. Figure 4 ,like Figure 1 The process shown can be further expanded to include the following steps:

[0071] S410: Acquire the region calibration information of the ROI and the region calibration information of the ROS located in the ROI.

[0072] This step can be considered as Figure 1 The principle of S110 shown in FIG. 1 is basically the same, and the area calibration information obtained in this step can be obtained by Figure 3 Calibrated by S310~S320 in.

[0073] S420: Obtain a detection region position set by performing a difference operation on a ROI position set represented by the ROI region calibration information and a ROS position set represented by the ROS region calibration information.

[0074] This step can be considered as Figure 1 A specific expansion of S120 is shown in FIG.

[0075] S430: Loading a mask base plate image having the same size as the target image, and obtaining a detection mask image having the same size as the target image by setting the area in the mask base plate image that overlaps with the detection area position set as a valid value.

[0076] This step can be considered as Figure 1 A specific expansion of S130 is shown in FIG.

[0077] S440: A composite image is obtained by performing a logical AND operation on aligned pixels of the detection mask image and the target image, wherein the composite image retains the original pixel array of the target image within the detection area and sets the pixel array outside the detection area to an invalid pixel array.

[0078] This step can be considered as Figure 1 The intersection operation in this step can be a logical AND operation of the aligned pixels of the detection mask image and the target image. That is, the valid value pixels in the detection mask image corresponding to the detection area are used to keep the original values ​​of the same-position pixels in the synthetic image obtained by the intersection operation; the invalid value pixels in the detection mask image corresponding to the pixels outside the detection area are used to set the same-position pixels in the synthetic image obtained by the intersection operation to invalid values.

[0079] In addition, the aforementioned category identification management mechanism and category identification selection mechanism also apply to Figure 4 The process shown.

[0080] Figure 5 For Figure 1 The image processing method shown is a schematic diagram of the extended process based on the identification of the number of regions. Figure 4 ,like Figure 1 The process shown can be further expanded to include the following steps:

[0081] S510: Acquire the region calibration information of the ROI and the region calibration information of the ROS located in the ROI.

[0082] This step can be considered as Figure 1 The principle of S110 shown in FIG. 1 is basically the same, and the area calibration information obtained in this step can be obtained by Figure 3 Calibrated by S310~S320 in.

[0083] S520: Detect the number of ROI position sets indicated by the ROI region calibration information and the number of ROS position sets indicated by the ROS region calibration information.

[0084] When the ROI region calibration information indicates a plurality of ROI position sets, and / or the ROS region calibration information indicates a plurality of ROS position sets, executing S530;

[0085] If the ROI region calibration information indicates a single ROI position set, and the ROS region calibration information indicates a single ROS position set, S530 is skipped and S540 is executed.

[0086] S530: performing a union operation on a plurality of ROI position sets and / or performing a union operation on a plurality of ROS position sets.

[0087] S540: Obtain the detection region position set by performing a difference operation on the ROI position set and the ROS position set. The ROI position set that performs a difference operation in S540 may be a single ROI position set represented by the ROI region calibration information, or may be a union operation result of multiple ROI position sets represented by the ROI region calibration information in S530; similarly, the ROS position set that performs a difference operation in S540 may be a single ROS position set represented by the ROS region calibration information, or may be a union operation result of multiple ROS position sets represented by the ROS region calibration information in S530.

[0088] The above steps S520 to S540 can be considered as follows: Figure 1 A specific extension of S120 shown in FIG. 1 and the difference operation in S540 can be similar to Figure 4 The principle of the S420 shown is basically the same.

[0089] S550: Loading a mask base plate image having the same size as the target image, and obtaining a detection mask image having the same size as the target image by setting the area in the mask base plate image that overlaps with the detection area position set as a valid value.

[0090] This step can be considered as Figure 1 A specific expansion of S130 is shown in FIG.

[0091] S560: A composite image is obtained by performing a logical AND operation on aligned pixels of the detection mask image and the target image, wherein the composite image retains the original pixel array of the target image within the detection area and sets the pixel array outside the detection area to an invalid pixel array.

[0092] This step can be considered as Figure 1 A specific extension of S140 shown in FIG. 140, and the intersection operation in this step can be combined with Figure 4 The principle of the S440 shown is basically the same.

[0093] In addition, the aforementioned category identification management mechanism and category identification selection mechanism also apply to Figure 5 The process shown.

[0094] When implementing the solution of the above embodiment, it is possible that the captured target image may have a posture deviation compared to the sample image used for setting the regional calibration information of ROI and ROS. In this case, it is necessary to introduce posture compensation to correct ROI and ROS to avoid the detection area failing to hit or completely hitting the area of ​​interest in the target image.

[0095] Figure 6 For Figure 1 The image processing method shown is based on the extended process diagram of regional pose correction. Figure 6 , in order to achieve the pose compensation of ROI and ROS, such as Figure 1 The process shown can be further expanded to include the following steps:

[0096] S610: Detecting boundary position conditions of ROI and ROS calibrated in a sample image in a human-computer interactive manner, and creating regional calibration information of ROI and ROS using the detected boundary position conditions, wherein the sample image and the target image present similar feature objects.

[0097] This step can be considered as Figure 3 The principles of S310 to S320 shown are basically the same.

[0098] S620: Detecting a reference feature calibrated in the sample image in a human-computer interaction manner, and detecting a calibration posture of the reference feature in the sample image, wherein the reference feature is located outside the range of the ROI and has a fixed position relationship with the ROI.

[0099] S630: Acquire the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI, and acquire the reference features and their calibration poses.

[0100] S640: Detect the posture deviation of the current posture of the reference feature in the target image relative to the marked posture information in the sample image.

[0101] This step can be achieved by means of feature detection.

[0102] S650: Perform posture correction on the regional calibration information of the ROI and ROS using the posture deviation.

[0103] The pose correction of the regional calibration information in this step can be considered as the pose correction of the ROI position set and the ROS position set.

[0104] S660: Determine a detection area according to the area calibration information of the ROI and ROS after posture correction, wherein the detection area is a reserved area after the overlapping area between the ROI position set and the ROS position set is removed.

[0105] This step can be considered as Figure 1 The principle of S120 shown is basically the same, and can also be further expanded as follows Figure 4 S420 as shown in or Figure 5 S520~S540 shown in.

[0106] S670: Create a detection mask image in which the effective area is constrained within the detection area.

[0107] This step can be considered as Figure 1 The principle of S130 shown is basically the same, and can also be further expanded as follows Figure 4 S430 as shown in or Figure 5 The S550 shown in FIG.

[0108] S680: Performing region validity filtering on the target image using the detection mask image.

[0109] This step can be considered as Figure 1 The principle of S140 shown is basically the same, and can also be further expanded as follows Figure 4 S440 as shown in or Figure 5 The S560 shown in FIG.

[0110] In addition, the aforementioned category identification management mechanism and category identification selection mechanism also apply to Figure 6 The process shown, for example, can further detect the sample classification instruction input in a human-computer interaction manner after S610, and associate the category identifier with the created ROI and ROS region calibration information, wherein the sample classification instruction includes the category identifier of the feature object presented in the sample image; can further associate the category identifier in the sample classification instruction with the detected reference feature after S620; can detect the region selection instruction input in a human-computer interaction manner at S630, and call the ROI and ROS region calibration information and the reference feature associated with the category identifier in the region selection instruction, wherein the region selection instruction includes the category identifier of the feature object presented in the target image to be processed.

[0111] Figure 7 For Figure 6 Schematic diagram of an example of calibration samples based on which the pose correction in the extended process is shown. Figure 8 For example Figure 7 Schematic diagram of an example of benchmark settings for the calibration samples shown.

[0112] Please see first Figure 7 The sample image 700 includes an object of interest 710 and an interfering object 720 located within the range of the object of interest 710 , and the sample image 700 also includes an additional object 730 having a fixed positional relationship with the object of interest 710 outside the range of the object of interest 710 .

[0113] Accordingly, please see Figure 8 In addition to setting the ROI 210 surrounding the object of interest 710 and the ROS 220 surrounding the interfering object 720 within the ROI 210 through S610 , a reference feature 800 aligned with the additional object 730 may be further set outside the ROI 210 through S620 .

[0114] Fig. 9 For the corresponding Figure 7 Schematic diagram of an example of a target sample offset from a calibration sample. Fig.10 For example Fig. 9 Schematic diagram of an example of pose correction of an offset target sample.

[0115] Please see first Fig. 9 , the target image 900 includes an object of interest 910, an interfering object 920 located within the range of the object of interest 910, and an additional object 930 outside the range of the object of interest 910 and having a fixed position relationship with the object of interest, and, compared to Figure 7 The sample image 700 shown is Fig. 9The target image 900 shown has an overall posture shift, so that the object of interest 910 , the interfering object 920 , and the additional object 930 in the target image 900 have posture deviations compared with the object of interest 710 , the interfering object 720 , and the additional object 730 in the sample image 700 , respectively.

[0116] Please see again Fig.10 , by detecting additional objects 930 in the target image 900, Figure 6 S640 in the method may determine a posture deviation of the current posture of the reference feature 800 in the target image 900 compared to the reference posture in the sample image 700, including coordinate deviations DeltaX and DeltaY, and an angle deviation DeltaA, and, Figure 6 S650 in the above method can use the coordinate deviations DeltaX and DeltaX in the pose deviation to coordinately translate the regional calibration information of ROI 210 and ROS 220, and then use the position coordinates of the reference feature 800 in the target image 900 as the center, and use the angle deviation DeltaA in the pose deviation to perform deflection calculation compensation DeltaX' and DeltaY' on the regional calibration information of ROI 210 and ROS 220 after the coordinate translation. The compensation amount of DeltaX' and DeltaY' can be determined according to the angle deviation DeltaA and the geometric position relationship between the reference feature 800 and the ROI 210 and ROS220.

[0117] It can be understood that the deflection angular calculation compensation based on the position coordinates of the reference feature in the target image and the angle deviation in the posture deviation can be performed before the coordinate translation based on the coordinate deviation in the posture deviation. Alternatively, the deflection angular calculation compensation based on the position coordinates of the reference feature in the target image and the angle deviation in the posture deviation, and the coordinate translation based on the coordinate deviation in the posture deviation can be performed one by one according to the actual posture deviation. That is, Figure 6 S640 shown in the figure may include: utilizing the coordinate deviation in the posture deviation to coordinately translate the regional calibration information of the ROI and ROS, and / or utilizing the angle deviation in the posture deviation to deflect and bend the regional calibration information of the ROI and ROS with the position coordinates of the reference feature in the target image as the center.

[0118] Therefore, by performing posture compensation on the regional calibration information of ROI 210 and ROS 220 with reference to the reference feature 800 , the local shielding effect of ROS 220 on ROI 210 can be more accurately targeted in the target image 900 .

[0119] Figures 11a to 11c Based on Figure 6Schematic diagram of an image processing example of the extended pipeline shown.

[0120] Please see first Fig.11a The sample image 100a is a sample image of a camera lens, and has a product model identifier on the front end surface of the camera lens, wherein the numeric portion in the product model identifier is an object of interest, and the numeric portion is doped with non-numeric interference characters, accordingly, ROI 1010 surrounds the numeric portion, ROS 1020 surrounds the non-numeric interference characters doped in the numeric portion, and in addition, the front end surface of the camera lens also has a logo portion, and the character I in the logo is set as a reference feature 1030.

[0121] Please see again Fig.11b , the target image 100b has a posture deviation compared to the sample image 100a. By using the posture deviation of the current posture of the reference feature 1030 in the target image 100b compared to the reference posture in the sample image 100a, the posture of the ROI 1010 and the ROS 1020 can be corrected, so that the ROI 1010 can surround the digital part in the target image 100b, and the ROS 1020 can surround the non-digital interference characters mixed in the digital part in the target image 100b.

[0122] Please see again Fig.11c , the detection area is determined by ROI 1010 and ROS 1020, a detection mask image is created by using the detection area, and then the detection mask image is used to perform regional validity filtering on the target image 100b, so that a synthetic image 100c can be obtained. The synthetic image 100c only retains the unadulterated digital part of the target image 100b located in the ROI 1010 and not blocked by the ROS 1020.

[0123] Fig.12 FIG. 1 is a schematic diagram of an exemplary structure of an image processing device in another embodiment. In another embodiment, an image processing device includes:

[0124] The calibration loading module 1210 is used to obtain the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI. For example, the regional calibration information obtained by the calibration loading module 1210 can represent the ROI position set and the ROS position set respectively, and the regional calibration information can include multiple ROI position sets and multiple ROS position sets.

[0125] The region definition module 1220 is used to determine the detection region according to the region calibration information, wherein the detection region is the reserved region after the overlapping region with the ROS in the ROI is removed. For example, the region definition module 1220 can obtain the detection region position set through the ROI position set represented by the region calibration information of the ROI and the ROS position set represented by the region calibration information of the ROS. For another example, when there are more than one ROI position sets represented by the region calibration information of the ROI, the region definition module 1220 can be further used to perform a union operation on the multiple ROI position sets represented by the region calibration information of the ROI, and / or perform a union operation on the multiple ROS position sets represented by the region calibration information of the ROS, and then obtain the detection region position set by performing a difference operation on the ROI position set and the ROS position set.

[0126] The mask creation module 1230 is used to create a detection mask image whose effective area is constrained to the detection area. For example, the mask creation module 1230 can load a mask base plate image having the same size as the target image, and obtain a detection mask image having the same size as the target image by setting the area in the mask base plate image that overlaps with the detection area position set as a valid value.

[0127] The image filtering module 1240 is used to perform regional validity filtering on the target image using the detection mask image. For example, the image filtering module 1240 can obtain a composite image by performing a logical AND operation on the aligned pixels of the detection mask image and the target image, wherein the composite image retains the original pixel array of the target image within the detection area and is set to an invalid pixel array outside the detection area.

[0128] Based on Fig.12 The image processing device shown can create a detection mask image using the detection area determined by the area calibration information of the ROI and the ROS. Since the detection area is defined in the non-overlapping area of ​​the ROI and the ROS, the detection mask image created using the detection area can have the effect of partially shielding the ROI using the ROS. Thus, by using the detection mask image whose effective area is constrained in the detection area to perform regional validity filtering on the target image, only the original image value in the local area of ​​the target image that is enclosed in the ROI and not shielded by the ROS can be retained, thereby eliminating internal interference and redundant image processing in the ROS, which helps to improve the image processing recognition accuracy and image processing efficiency of the target image.

[0129] Fig.13 FIG11 is a schematic diagram of an extended structure of the image processing device shown in FIG11. Fig.13 , in order to support the manual setting of ROI and ROS, such as Fig.12The image processing device shown may further include a calibration creation module 1200, which is used to detect the boundary position conditions of the ROI and ROS calibrated in the sample image in a human-computer interactive manner, and create regional calibration information of the ROI and ROS using the detected boundary position conditions, wherein the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size. For example, the boundary position conditions detected by the calibration creation module 1200 may include discrete boundary features such as edge points. And the calibration creation module 1200 may fit the discrete boundary features into a closed polygon with the boundary points as vertices, and use the fitted closed polygon to create regional calibration information representing the position set.

[0130] If the aforementioned category identification management mechanism and category identification selection mechanism are introduced, the calibration creation module 1200 can be further used to detect sample classification instructions input in a human-computer interaction manner, and associate the category identification with the created ROI and ROS regional calibration information, wherein the sample classification instruction includes the category identification of the feature object presented in the sample image; accordingly, the calibration loading module 1210 can be further used to detect the region selection instruction input in a human-computer interaction manner, and call the ROI and ROS regional calibration information associated with the category identification in the region selection instruction, wherein the region selection instruction includes the category identification of the feature object presented in the target image to be processed.

[0131] Fig.14 FIG11 is another schematic diagram of an extended structure of the image processing device shown in FIG11. Fig.14 , in order to support the pose correction of ROI and ROS, such as Fig.12 The image processing device shown may further include a posture compensation module 1250, which is used to obtain the reference feature and its calibration posture, detect the posture deviation of the current posture of the reference feature in the target image relative to the calibration posture in the sample image, and use the posture deviation to perform posture correction on the regional calibration information of the ROI and ROS. For example, the posture compensation module 1250 can use the coordinate deviation in the posture deviation to perform coordinate translation on the regional calibration information of the ROI and ROS, and / or, with the position coordinates of the reference feature in the target image as the center, use the angle deviation in the posture deviation to perform deflection and bending compensation on the regional calibration information of the ROI and ROS after the coordinate translation.

[0132] Still see Fig.14 ,The datum feature setting can also support manual methods, such as Fig.12 The image processing apparatus shown may further include a reference calibration module 1260 for detecting reference features calibrated in the sample image in a human-computer interactive manner, and detecting the calibration pose of the reference features in the sample image.

[0133] It is understandable that if Fig.14 The structure shown can also be further included in Fig.13 The calibration creation module 1200 shown in .

[0134] If the aforementioned category identifier management mechanism and category identifier selection mechanism are introduced, the calibration creation module 1200 can be further used to associate the category identifier in the sample classification instruction with the calibrated reference feature; accordingly, the posture compensation module 1250 can be further used to detect the area selection instruction input in a human-computer interaction manner, and call the reference feature associated with the category identifier in the area selection instruction.

[0135] Fig.15 FIG. 1 is a schematic diagram of a detection device in another embodiment. Fig.15 The detection device in this embodiment may include a processor 1510, a memory 1520, and a non-transitory computer-readable storage medium 1530, wherein the processor 1510 is used to execute the following Figures 3 to 6 The steps of the image processing method shown in any one of the items; the memory 1520 is an optional component, which can store the calibration area information of the pre-calibrated ROI and ROS, as well as the reference features and their reference poses in the sample image; the non-transitory computer-readable storage medium 1530 can store instructions, which when executed by the processor 1510 can enable the processor 1510 to perform the following Figures 3 to 6 The steps in any one of the image processing methods shown in .

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire regional calibration information of a region of interest (ROI) and regional calibration information of a shielding region (ROS) located in the ROI; wherein the regional calibration information of the ROI is used to indicate a set of coordinate positions covered by the ROI in an image coordinate system, and the regional calibration information of the ROS is used to indicate a set of coordinate positions covered by the ROS in an image coordinate system, and the regional calibration information of the ROI and the ROS is set using a sample image, and the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size; Performing posture correction on the regional calibration information of the ROI and the ROS using the posture deviation of the current posture of the detected reference feature in the target image relative to the calibration posture of the reference feature in the sample image; wherein the reference feature is aligned with the additional object in the sample image, and the additional object has a fixed position relationship with the object of interest surrounded by the ROI in the sample image; Determine the detection area according to the regional calibration information of the ROI and the ROS; wherein the detection area is the reserved area after the overlapping area between the ROI and the ROS is removed; Creating a detection mask image whose effective area is constrained within the detection area; The detection mask image is used to perform region validity filtering on the target image.

2. The image processing method according to claim 1, characterized in that: Before obtaining the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI, the method further includes: Detect the boundary position conditions of ROI and ROS marked in the sample image in a human-computer interactive manner; The detected boundary position conditions are used to create the regional calibration information of ROI and ROS.

3. The image processing method according to claim 1, characterized in that: Before obtaining the regional calibration information of the ROI and the regional calibration information of the ROS located in the ROI, the method further includes: detecting a sample classification instruction input in a human-computer interaction manner, and associating a category identifier with the created regional calibration information of the ROI and the ROS, wherein the sample classification instruction includes a category identifier of a feature object presented in the sample image; Acquiring the region calibration information of the ROI and the region calibration information of the ROS located in the ROI includes: detecting a region selection instruction input in a human-computer interaction manner, and calling the region calibration information of the ROI and the ROS associated with the category identifier in the region selection instruction, wherein the region selection instruction includes the category identifier of the feature object presented in the target image to be processed.

4. The image processing method according to claim 1, characterized in that: Determining the detection area according to the area calibration information includes: obtaining a detection area position set by performing a difference operation on a ROI position set represented by the area calibration information of the ROI and a ROS position set represented by the area calibration information of the ROS; Creating a detection mask image whose effective area is constrained within the detection area includes: loading a mask base image having the same size as the target image, and obtaining a detection mask image having the same size as the target image by setting an area in the mask base image that overlaps with the detection area position set as a valid value; Using a detection mask image to perform regional validity filtering on a target image includes: obtaining a composite image by performing a logical AND operation on aligned pixels of the detection mask image and the target image, wherein the composite image retains the original pixel array of the target image within the detection area and is set to an invalid pixel array outside the detection area.

5. The image processing method according to claim 4, characterized in that: When the region calibration information of the ROI represents a plurality of ROI position sets, determining the detection region according to the region calibration information further comprises: performing a union operation on the plurality of ROI position sets before performing a difference operation on the ROI position set and the ROS position set; When the ROS region calibration information indicates a plurality of ROS position sets, determining the detection region according to the region calibration information further includes: performing a union operation on the plurality of ROS position sets before performing a difference operation on the ROI position set and the ROS position set.

6. The image processing method according to claim 1, characterized in that: Before performing posture correction on the regional calibration information of the ROI and the ROS by using the posture deviation of the current posture of the detected reference feature in the target image relative to the calibration posture, the method further includes: Detecting fiducial features calibrated in sample images in a human-computer interactive manner; Detect the reference features’ localized poses in sample images.

7. The image processing method according to claim 1, characterized in that: The pose deviation of the current pose of the detected reference feature in the target image relative to the calibration pose is used to correct the pose of the regional calibration information of ROI and ROS, including: Using the coordinate deviation in the pose deviation to coordinate translate the regional calibration information of the ROI and ROS; and / or Taking the position coordinates of the reference feature in the target image as the center, the angle deviation in the pose deviation is used to perform deflection and bending compensation on the regional calibration information of ROI and ROS.

8. An image processing device, characterized in that: include: A calibration loading module is used to obtain the regional calibration information of the region of interest ROI and the regional calibration information of the shielding area ROS located in the ROI; The region calibration information of ROI is used to indicate the set of coordinate positions covered by ROI in the image coordinate system, and the region calibration information of ROS is used to indicate the set of coordinate positions covered by ROS in the image coordinate system. The region calibration information of ROI and ROS is set using a sample image, and the sample image and the target image present the same type of feature objects, and the sample image and the target image have the same size; a pose compensation module for using a pose deviation of a detected current pose of a reference feature in a target image relative to a calibration pose of the reference feature in a sample image, wherein the reference feature is aligned with an additional object in the sample image, and the additional object has a fixed positional relationship with an object of interest surrounded by the ROI in the sample image; A region definition module is used to determine the detection region according to the region calibration information of the ROI and the ROS; wherein the detection region is the reserved region after the overlapping region of the ROI and the ROS is removed; A mask creation module, used for creating a detection mask image whose effective area is constrained within the detection area; The image filtering module is used to perform regional validity filtering on the target image using the detection mask image.

9. The image processing device according to claim 8, characterized in that: Further including: The calibration creation module is used to detect the boundary position conditions of the ROI and ROS calibrated in the sample image in a human-computer interaction manner, and create regional calibration information of the ROI and ROS using the detected boundary position conditions.

10. The image processing device according to claim 8, characterized in that: The calibration creation module is further used to detect a sample classification instruction input in a human-computer interaction manner, and associate a category identifier with the created ROI and ROS regional calibration information, wherein the sample classification instruction includes a category identifier of a feature object presented in the sample image; The calibration loading module is further used to detect the region selection instruction input in a human-computer interaction manner, and call the region calibration information of the ROI and ROS associated with the category identifier in the region selection instruction, wherein the region selection instruction contains the category identifier of the feature object presented in the target image to be processed.

11. The image processing device according to claim 8, characterized in that: Further including: The reference calibration module is used to detect the reference features calibrated in the sample image in a human-computer interactive manner, and to detect the calibration posture of the reference features in the sample image.

12. A detection device, characterized in that: The method comprises a processor configured to execute the steps in the image processing method according to any one of claims 1 to 7.

13. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores instructions which, when executed by a processor, cause the processor to perform the steps in the image processing method according to any one of claims 1 to 7.

Images