On-device target object detection method and apparatus, electronic device, and storage medium

Abstract

The present application relates to the technical field of image recognition, and an on-device target object detection method, device, electronic equipment and storage medium are provided.The on-device target object detection method comprises: acquiring a to-be-detected image of a device containing a target object, the target object comprising components on the device and assembly gaps between the components; segmenting the to-be-detected image to obtain a plurality of component images, each component image not containing the assembly gaps; determining an outer polygonal region of a corresponding component image based on the component images, wherein the extension direction of the side of the outer polygonal region close to the assembly gaps is parallel to the length direction of the assembly gaps determined in advance; and determining the assembly gaps in the to-be-detected image based on the determined outer polygonal regions.The embodiments provided by the present application avoid direct detection of the assembly gaps, and improve the detection accuracy of the assembly gaps.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a target object detection method on a device, a target object detection device on a device, an electronic device and a storage medium. BACKGROUND

[0002] The visual detection device is one of the common detection devices. The visual detection device can acquire images of a device to be detected at different angles through multiple cameras, and implement detection of the device to be detected through different detection algorithms. Nowadays, in many production test scenes, there is a higher requirement for the rate (throughput) of device detection, and the sorting and discharging of good products and defective products need to be performed immediately after the detection is completed, which requires the algorithm to complete the detection of the assembly gap of the device in an image in a very short time, so as to determine whether the device is a good product or a defective product before discharging, and thus the device can be stored in the correct box when discharged. Taking an inductor device as an example, the gap between the winding part and the cover plate part after assembly is an important parameter, and if the gap between the winding part and the cover plate part is too large, the electrical performance of the inductor device will be affected.

[0003] Although the existing image processing algorithm has the ability to detect the assembly gap, these existing algorithms often need to perform edge detection, boundary fitting, or recognition of specific objects such as gaps. Whether it is a fitting algorithm or an object recognition algorithm, there are problems of large amount of calculation, high resource occupation, long time consumption or complex parameter adjustment, and it is not suitable for the scene of real-time detection and completion of calculation and output of results in a short time. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a target object detection method, device, electronic device and storage medium on a device to solve the above problems.

[0005] In order to achieve the above purpose, the first aspect of the present application provides a target object detection method on a device, comprising: acquiring a to-be-detected image of a device containing a target object, the target object including assembly gaps between components on the device; segmenting the to-be-detected image to obtain a plurality of component images, each component image not containing the assembly gap; determining a circumscribed polygon region of the corresponding component image based on the component image; wherein the extension direction of the edge of the circumscribed polygon region close to the assembly gap is parallel to the length direction of the assembly gap determined in advance; and determining the assembly gap in the to-be-detected image based on the determined circumscribed polygon region.

[0006] Preferably, the method further comprises: performing morphological closing operation on the circumscribed polygon region corresponding to the assembly gap to be detected to obtain a first image region; and performing difference operation on the obtained first image region and the circumscribed polygon before the morphological closing operation to determine the assembly gap in the image to be detected corresponding to the circumscribed polygon region.

[0007] Preferably, the circumscribed polygon is a minimum envelope rectangle of the component in the image to be detected.

[0008] Preferably, before the image to be detected is segmented to obtain a plurality of component images, the method further comprises: determining a plurality of target regions from the image to be detected, wherein the target regions include the assembly gap; and segmenting the target regions to obtain a plurality of component images.

[0009] Preferably, the method further comprises: determining an offset between a device in the image to be detected and a device in a preset template image, wherein the preset template image includes a plurality of regions of interest, and the regions of interest include the assembly gap; and generating regions corresponding to the regions of interest in the image to be detected according to the preset template image and the offset to obtain the target regions.

[0010] Preferably, before the target regions are segmented to obtain a plurality of component images, the method further comprises: performing filtering processing on the target regions in a direction parallel to the assembly gap.

[0011] Preferably, after the target regions are segmented to obtain a plurality of component images, the method further comprises: performing direction correction on the plurality of component images to rotate the plurality of component images to a preset direction.

[0012] Preferably, after the image to be detected is segmented to obtain a plurality of component images, the method further comprises: performing denoising operation on the component images.

[0013] Preferably, the denoising operation on the component images comprises: performing hole filling on the component images; and / or performing morphological opening operation on the component images and performing closing operation based on the result of the opening operation.

[0014] Preferably, if the target regions are multiple; each target region is arranged at intervals along the length direction of the assembly gap, and the determining the assembly gap in the image to be tested based on the determined circumscribed polygonal region comprises: in each target region, determining a corresponding assembly gap based on the determined circumscribed polygonal region; and determining the assembly gap in the image to be tested based on the multiple assembly gaps determined in the multiple target regions.

[0015] Preferably, after the assembly gap in the image is obtained, the method further comprises: determining a detection result of the device according to the characteristics of the assembly gap.

[0016] In a second aspect of the present application, a device target object detection device is also provided, comprising: an image acquisition module configured to acquire an image to be tested of a device containing a target object, the target object comprising components on the device and assembly gaps between the components; an image segmentation module configured to segment the image to be tested to obtain multiple component images, each of the component images not containing the assembly gaps; a region determination module configured to determine a circumscribed polygonal region of a corresponding component image based on the component image, wherein the extension direction of the edge of the circumscribed polygonal region close to the assembly gap is parallel to the length direction of the assembly gap determined in advance; and a gap determination module configured to determine the assembly gap in the image to be tested based on the determined circumscribed polygonal region.

[0017] Preferably, the determining the assembly gap in the image to be tested based on the determined circumscribed polygonal region comprises: performing a morphological closing operation on the circumscribed polygonal region corresponding to the assembly gap to be detected to obtain a first image region; and performing a difference operation on the obtained first image region and the circumscribed polygonal region before the morphological closing operation to determine the assembly gap in the image to be tested corresponding to the corresponding circumscribed polygonal region.

[0018] Preferably, the circumscribed polygonal region is the minimum outer envelope rectangle of the component in the image to be tested.

[0019] Preferably, before the segmenting the image to be tested to obtain multiple component images, it comprises: determining a plurality of target regions from the image to be tested, the target regions containing the assembly gaps; and the segmenting the image to be tested to obtain multiple component images comprises: segmenting the target regions to obtain multiple component images.

[0020] Preferably, the determining the target region from the to-be-tested image comprises: determining a shift between a device in the to-be-tested image and a device in a preset template image, the preset template image containing a plurality of regions of interest, the regions of interest including the assembly gap; and generating regions corresponding to the regions of interest in the to-be-tested image according to the preset template image and the shift, to obtain the target region.

[0021] Preferably, before the segmenting the target region to obtain a plurality of component images, the device further comprises: performing filtering processing on the target region in a direction parallel to the assembly gap.

[0022] Preferably, after the segmenting the target region to obtain a plurality of component images, the device further comprises: performing direction correction on the plurality of component images, so that the plurality of component images are rotated to a preset direction.

[0023] Preferably, after the segmenting the to-be-tested image to obtain a plurality of component images, the device further comprises: performing a denoising operation on the component images.

[0024] Preferably, the denoising operation on the component images comprises: performing hole filling on the component images; and / or performing a morphological opening operation on the component images, and performing a closing operation based on a structure of the opening operation.

[0025] Preferably, if the target region is a plurality of target regions, each target region is arranged at intervals along a length direction of the assembly gap, and the determining the assembly gap in the to-be-tested image based on the determined circumscribed polygon region comprises: determining a corresponding assembly gap in each target region based on the determined circumscribed polygon region; and determining the assembly gap in the to-be-tested image based on a plurality of assembly gaps determined in the plurality of target regions.

[0026] Preferably, after the determining the assembly gap in the image, the device further comprises: determining a detection result of the device according to a feature of the assembly gap.

[0027] In a third aspect of the present application, a position determination device of a target object is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the position determination method of the target object when executing the computer program.

[0028] In a fourth aspect of the present application, a computer readable storage medium is provided, which stores instructions, and when the instructions are executed on a computer, the computer executes the steps of the position determination method of the target object.

[0029] The fifth aspect of the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the aforementioned target object position determination method.

[0030] The technical solution has at least the following beneficial effects:

[0031] (1) The embodiment of the present application obtains a plurality of component images by segmenting the to-be-measured image, then determines the corresponding circumscribed polygon region based on the component image, and determines the assembly gap based on the circumscribed polygon region. The present application does not need to perform edge detection, boundary fitting, or identification of specific objects such as gaps, and only performs region processing, so it does not use complex calculation methods, and therefore has high calculation efficiency, saves calculation resources, and can be applied to high-throughput scenarios. And because the present application does not directly process the boundary region, it can avoid the adverse effects of edge noise of the measured object on the detection result.

[0032] (2) In the present application, the component image is obtained by image segmentation, and the area of the component image is large, so the signal-to-noise ratio is high, which can stabilize the component image. The existing technology directly extracts the assembly gap, and the assembly gap region is small, so the anti-interference effect is poor and it is not easy to process. Therefore, compared with directly extracting the assembly gap, the present application extracts a larger and more recognizable target, has the advantage that the target is easier to extract, and can effectively avoid the influence of noise in the image on the processing result.

[0033] (3) The embodiment of the present application can be flexibly extended, and a plurality of regions of interest are used to segment to measure the assembly gap, which is more in line with the actual shape of the assembly gap, and can improve the detection accuracy of irregular assembly gaps.

[0034] Other features and advantages of the embodiments of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings are included to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used together with the following specific embodiments to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the drawings:

[0036] Figure 1 The steps of the device target object detection method according to the embodiment of the present application are schematically shown;

[0037] Figure 2 The schematic diagram of determining the assembly gap according to the circumscribed polygon in the embodiment of the present application is schematically shown;

[0038] Figure 3A schematic diagram illustrating determination of assembly gap in multiple target regions according to an embodiment of the present application is shown schematically.

[0039] Figure 4 A structural schematic diagram of a device target object detection apparatus according to an embodiment of the present application is shown schematically. DETAILED DESCRIPTION

[0040] The specific embodiments of the present application are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to illustrate and explain the present application, and are not intended to limit the present application.

[0041] Figure 1 A schematic diagram of steps of a device target object detection method according to an embodiment of the present application is shown schematically. As shown in Figure 1 A device target object detection method includes:

[0042] S01, obtaining a to-be-detected image of a device containing a target object, the target object including components on the device and assembly gap between the components; taking an inductor as an example, it generally includes a cover plate portion and a winding portion. The winding portion and the cover plate portion are assembled together by adhesive, and the gap between the winding portion and the cover plate portion after assembly is an important parameter. Here, the winding portion and the cover plate portion both belong to components on the inductor, and the assembly gap between them is the target object to be identified.

[0043] Exemplarily, the to-be-detected image can be an image of any device containing an assembly gap, and the to-be-detected image can be a static image or any video frame in a dynamic video. The to-be-detected image can be an original image collected by an image collection device, or an image obtained after pre-processing (such as digitization, normalization, smoothing, etc.) of the original image collected by the image collection device. It can be understood that the pre-processing of the original image can include the operation of extracting a sub-image containing the part of the assembly gap and the components forming the assembly gap from the original image collected by the image collection device to obtain the to-be-detected image.

[0044] S02, segmenting the image to be tested to obtain a plurality of component images, each of the component images not containing the assembly gap; in the prior art, edge detection and other algorithms are used to identify specific objects such as assembly gaps, but there are problems in the background technology. Due to the many differences between the components and the assembly gap in imaging, the identification of the component image is mainly used in this step, and a plurality of component images not containing the assembly gap are obtained by image segmentation. The method of segmenting the image to be tested in this step includes but is not limited to: threshold-based segmentation method (binaryzation), region growing and splitting and merging method, edge-based segmentation method, wavelet transform-based segmentation method, neural network-based segmentation method, etc. The threshold-based segmentation method (binaryzation) is the most efficient way.

[0045] It can be understood that after segmenting the image to be tested, a plurality of component images are obtained, and the components in the plurality of component images are used to form the target object (assembly gap); this step mainly extracts the images of the components except the assembly gap. For example, when using threshold segmentation, the gray difference between the components and the gap in the image is used to extract the component image. For example, a gray threshold can be set, and the area greater than the gray threshold is the extracted component image.

[0046] S03, determining the circumscribed polygon region of the corresponding component image based on the component image; wherein the extension direction of the side of the circumscribed polygon region close to the assembly gap is parallel to the length direction of the assembly gap determined in advance. According to the component image obtained in the previous step, image operation is performed to obtain the circumscribed polygon region including the component image. The shape of the circumscribed polygon region is not limited here, but the direction of one side of the circumscribed polygon region needs to be consistent with the length direction of the assembly gap. For example, the length direction of the assembly gap determined in advance is the horizontal direction, and the extension direction of the side of the circumscribed polygon region close to the assembly gap should also be the horizontal direction.

[0047] S04, determining the assembly gap in the image to be tested based on the determined circumscribed polygon region. Through image operation on the circumscribed polygon region, the region formed between the circumscribed polygon regions is taken as the assembly gap. In some possible embodiments, there can also be a step of combining a plurality of regions obtained into one assembly gap.

[0048] The embodiment of the present application obtains a plurality of component images by segmenting the to-be-tested image, then determines the corresponding circumscribed polygon region based on the component images, and determines the assembly gap based on the circumscribed polygon region. Since the present application does not need to perform edge detection, boundary fitting, or identification of specific objects such as gaps, and only performs region processing, the present application does not use complex calculation methods, and therefore has high calculation efficiency, saves calculation resources, and can be applied to high-throughput scenarios. Moreover, since the present application does not directly process the boundary region, the adverse effects of edge noise of the measured object on the detection result can be avoided.

[0049] In the above embodiment, the component image is extracted by taking advantage of the characteristics of a large area and high signal-to-noise ratio of the component image. Since the embodiment does not directly extract the assembly gap, the disadvantages of a small assembly gap region, poor anti-interference effect, and difficulty in processing are avoided. That is, direct extraction of the gap region is greatly affected by image noise, resulting in inaccurate processing results.

[0050] In some optional embodiments, the determination of the assembly gap in the to-be-tested image based on the determined circumscribed polygon includes: performing a morphological closing operation on the circumscribed polygon region corresponding to the assembly gap to be detected to obtain a first image region; and performing a difference operation on the obtained first image region and the circumscribed polygon before the morphological closing operation to determine the assembly gap in the to-be-tested image corresponding to the circumscribed polygon region. Figure 2 A schematic diagram for determining an assembly gap based on a circumscribed polygon according to an embodiment of the present application is schematically shown. As shown in Figure 2 The component image is the component image identified in step S02. Since the component image can be irregular, the circumscribed polygon region of the component image is calculated, and two circumscribed polygon regions are shown in the figure, and there is a gap between the two circumscribed polygon regions. A first image region is obtained by a morphological closing operation, that is, the region shown by the largest rectangle in the periphery. The first image region includes the two circumscribed polygon regions and the gap region therebetween. Then, a difference operation is performed on the first image region (the largest rectangle in the figure) and the circumscribed polygon (the two spaced circumscribed polygons) before the morphological closing operation to obtain the assembly gap shown by the shaded part in the figure. The above describes each region as a rectangle only for ease of description, and in fact, the circumscribed polygon region needs to be determined according to the shape of the component itself.

[0051] In some optional embodiments, the circumscribed polygon is the minimum circumscribed rectangle of the component in the to-be-tested image. The circumscribed rectangle is fitted because the body region where the component is located can be incomplete due to various interferences, and in particular, the edge near the gap can present a pit, which affects the accuracy of measurement, so the body region is fitted into a circumscribed rectangle.

[0052] In some optional embodiments, before segmenting the image under test to obtain multiple component images, the method includes: determining a plurality of target regions from the image under test, wherein the target regions include the assembly gap; the segmentation of the image under test to obtain multiple component images includes: segmenting the target regions to obtain multiple component images. Unlike the previous embodiment, which considers the entire device image under test as a single target region, this embodiment determines multiple target regions from the entire device and segments each target region to obtain a component image within that target region. Figure 3 A schematic diagram illustrating the determination of assembly gaps in multiple target regions according to an embodiment of the present invention is shown. Figure 3 As shown, taking a target area of ​​3 as an example, target area 1, target area 2, and target area 3 are identified in the image to be tested. Each target area includes a component image and an assembly gap. The assembly gap within each target area is identified according to the aforementioned steps. Finally, the assembly gap in the entire device image to be tested is obtained based on the multiple obtained assembly gaps. This implementation is suitable for scenarios with irregular assembly gaps, avoiding significant differences between the identification results of irregular assembly gaps and their actual shapes, thereby improving the detection accuracy of irregular assembly gaps. Taking a conical assembly gap as an example, if only one target area is identified, a rectangular assembly gap area will be obtained, resulting in a large error. However, if multiple target areas are identified, multiple rectangular areas with different widths will be obtained, thus more closely approximating the conical shape of the assembly gap.

[0053] In some optional embodiments, the determining the target region from the to-be-tested image comprises: determining an offset between the device in the to-be-tested image and the device in the preset template image, the preset template image containing a plurality of regions of interest (ROI), the region of interest including the assembly gap; and generating a region corresponding to the region of interest in the to-be-tested image according to the preset template image and the offset, to obtain the target region. Specifically, in an actual detection process, the device is likely to be offset, for example, although the camera for shooting the template image and the camera for shooting the to-be-tested image have no change in position, the device is likely to be offset (translation or rotation) in the to-be-tested image due to the fact that the device is not positioned in place or moves during transmission. If the target region corresponding to the ROI in the to-be-tested image is generated directly according to the position of the ROI in the template image, the target region is likely to fail to cover the position to be concerned in the current detection, resulting in an error in the detection result. In order to avoid the above problem, the initial position of the target region in the to-be-tested image is the same as the position of the ROI in the template image, and the position of the target region in the to-be-tested image needs to be corrected according to the position offset (the above offset) between the device in the to-be-tested image and the device in the template image, so that the corrected target region can cover the correct position of the device in the to-be-tested image. That is, the relative positional relationship between the ROI and the device in the template image is the same as the relative positional relationship between the target region and the device in the to-be-tested image. The steps are as follows: Step 1, determining the offset between the device in the to-be-tested image and the device in the template image; wherein the current position can be determined by template matching, and the offset is calculated according to the deviation (mainly for the outlines of the devices in the two images) between the current position and the template position. Step 2, generating the target region corresponding to the ROI in the to-be-tested image according to the ROI in the template image, and adjusting the position of the target region in the image according to the offset, wherein the ROI is a rotatable rectangle used to mark the length direction of the assembly gap, which is the expected gap direction, so that the target region covers the corresponding region of the to-be-tested image. Through the above operation, the relative positional relationship between the target region and the device in the to-be-tested image is the same as the relative positional relationship between the ROI and the device in the template image after the target region is adjusted based on the offset.

[0054] In some optional embodiments, before the target region is segmented to obtain a plurality of component images, the method further comprises: performing filtering processing on the target region in a direction parallel to the assembly gap. Mean filtering is performed along the direction of the ROI in the target region to filter a part of interference and noise. Filtering along the direction of the ROI can maximize the contrast features of the gap edge while eliminating noise. The filtering method used here can also be weighted filtering or median filtering, etc.

[0055] In some optional embodiments, after the target region is segmented to obtain a plurality of component images, the method further comprises: performing direction correction on the plurality of component images, so that the plurality of component images are rotated to a preset direction. The direction correction on the extracted body region ensures that the direction of the assembly gap is 0 degrees. Specifically, the template image matching obtains the angle of the current to-be-tested image, generates a rotation transformation matrix, and performs rotation transformation on the body region to 0 degrees. In this direction, the direction of the target region in the to-be-tested image is the 0-degree direction, i.e., the horizontal direction. For example, if the direction of the target region in the to-be-tested image is offset by 30 degrees relative to the horizontal direction, the ROI center is rotated by 30 degrees, so that the direction of the target region in the to-be-tested image is the horizontal direction. The adjustment in the 0-degree direction in this step is mainly to effectively reduce the complexity of the calculation and improve the efficiency of the calculation.

[0056] In some optional embodiments, after the to-be-tested image is segmented to obtain a plurality of component images, the method further comprises: performing a denoising operation on the component images. This step is mainly to obtain a more accurate component image due to the existence of foreign matter or dirt on the components constituting the target object (assembly gap), which causes the gray scale to be darker and thus is not recognized as a component. In summary, the main purpose of the denoising operation is to obtain a more accurate component image.

[0057] In some optional embodiments, the denoising operation on the component images comprises: performing hole filling on the component images; and performing hole filling on the component images after the direction correction, to further eliminate noise interference. Since there may be regions with low gray scale values in the component images obtained after the segmentation such as binarization, these regions cannot be extracted by binarization. Therefore, these regions need to be extracted as part of the component images by hole filling. For example, the region that meets the requirements after binarization is 1, and the value of the hole region is 0. This step needs to fill the hole region to 1.

[0058] The foregoing denoising operation can further comprise: performing morphological opening operation on the component images, and performing closing operation based on the result of the opening operation. The morphological opening operation is performed on the body region in the 0-degree direction to eliminate the influence of foreign matter or glue inside the assembly gap. This step is mainly to eliminate the interference of the foreign matter or glue inside the assembly gap, which may be misidentified as a component image due to the existence of foreign matter or glue at the assembly gap, which should be darker. The morphological opening operation is performed on the body region to eliminate such interference. The morphological closing operation is performed on the image after the morphological opening operation to eliminate the influence of foreign matter or dirt on the body.

[0059] In some optional embodiments, if the target region is multiple, each target region is arranged along the length direction of the assembly gap; the method further comprises: determining the assembly gap in the image based on the multiple assembly gaps determined in the multiple target regions. Figure 3 As shown in the figure, when the target region is multiple, the multiple target regions are arranged along the length direction of the assembly gap; that is, when the length direction of the assembly gap is the horizontal direction, the multiple target regions should be arranged along the horizontal direction. For example, when the gap region is conical, a single target region cannot accurately reflect the shape of the assembly gap, and multiple rectangular regions with different widths can better fit the assembly gap. When the multiple target regions are all identified by the ROIs in the template image, the ROIs in the template should be correspondingly set, and the number of ROIs in the template image has a corresponding relationship with the number of target regions.

[0060] In some optional embodiments, after the assembly gap in the image is obtained, the method further comprises: determining the detection result of the device according to the characteristics of the assembly gap. The characteristics of the assembly gap include but are not limited to the width size and the shape characteristics, etc. For example, an excessively large assembly gap is a relatively common defect, and when the assembly gap obtained by the foregoing steps is greater than a preset threshold, it can be determined that the detection result of the device is unqualified. In the case where multiple assembly gaps are determined by multiple target regions, as long as one of the assembly gaps exceeds the control specification, the evaluation result of the device can be obtained.

[0061] Based on the same inventive concept, the embodiments of the present application also provide a device target object detection device. Figure 4 The structure of the device target object detection device according to the embodiments of the present application is schematically shown. As shown in the figure, Figure 4 The device comprises: an image acquisition module, configured to acquire a to-be-tested image of a device containing a target object, the target object comprising components on the device and assembly gaps between the components; an image segmentation module, configured to segment the to-be-tested image to obtain multiple component images, each component image not containing the assembly gap; a region determination module, configured to determine a circumscribed polygonal region of a corresponding component image based on the component image, wherein the extension direction of the edge of the circumscribed polygonal region close to the assembly gap is parallel to the length direction of the assembly gap determined in advance; and a gap determination module, configured to determine the assembly gap in the to-be-tested image based on the determined circumscribed polygonal region.

[0062] In some optional embodiments, the determining the assembly gap in the to-be-tested image based on the determined bounding polygon comprises: performing a morphological closing operation on the bounding polygon region corresponding to the assembly gap to be detected to obtain a first image region; and performing a difference operation on the obtained first image region and the bounding polygon before the morphological closing operation to determine the assembly gap in the to-be-tested image corresponding to the bounding polygon region.

[0063] In some optional embodiments, the bounding polygon is a minimum envelope rectangle of the component in the to-be-tested image.

[0064] In some optional embodiments, before the obtaining the plurality of component images by segmenting the to-be-tested image, the method further comprises: determining a plurality of target regions in the to-be-tested image, wherein the target regions include the assembly gap; and the obtaining the plurality of component images by segmenting the to-be-tested image comprises: obtaining the plurality of component images by segmenting the target regions.

[0065] In some optional embodiments, the determining the plurality of target regions in the to-be-tested image comprises: determining an offset between a device in the to-be-tested image and a device in a preset template image, wherein the preset template image includes a plurality of regions of interest, and the regions of interest include the assembly gap; and generating regions corresponding to the regions of interest in the to-be-tested image according to the preset template image and the offset to obtain the target regions.

[0066] In some optional embodiments, before the obtaining the plurality of component images by segmenting the target regions, the apparatus further comprises: performing filtering processing on the target regions in a direction parallel to the assembly gap.

[0067] In some optional embodiments, after the obtaining the plurality of component images by segmenting the target regions, the apparatus further comprises: performing direction correction on the plurality of component images to rotate the plurality of component images to a preset direction.

[0068] In some optional embodiments, after the obtaining the plurality of component images by segmenting the to-be-tested image, the apparatus further comprises: performing a denoising operation on the component images.

[0069] In some optional embodiments, the performing the denoising operation on the component images comprises: performing hole filling on the component images; and / or performing a morphological opening operation on the component images and performing a closing operation based on a result of the opening operation.

[0070] In some optional embodiments, if the target region is multiple; each target region is arranged at intervals along the length direction of the assembly gap, and the determination of the assembly gap in the image to be tested based on the determined circumscribed polygon region comprises: in each target region, the corresponding assembly gap is determined based on the determined circumscribed polygon region; and the assembly gap in the image to be tested is determined based on the multiple assembly gaps determined in the multiple target regions.

[0071] In some optional embodiments, after the assembly gap in the image is obtained, the device further comprises: determining the detection result of the device according to the characteristics of the assembly gap.

[0072] In some embodiments provided by the present application, an electronic device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned device target object detection method when executing the computer program. The processor herein has the functions of numerical calculation and logical operation, and has at least a central processing unit (CPU) with data processing capability, a random access memory (RAM), a read-only memory (ROM), various I / O ports, and an interrupt system, etc. The processor includes a core, and the core retrieves corresponding program units from the memory. The core can be one or more, and the above-mentioned method is realized by adjusting the core parameters. The memory can include a non-persistent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory, etc. in the form of a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0073] In an embodiment of the present application, a storage medium is also provided, and the storage medium stores instructions, which, when executed on a computer, cause the processor to be configured to perform the steps of the above-mentioned device target object detection method.

[0074] In an embodiment provided by the present application, a computer program product is provided, which includes a computer program, and the computer program, when executed by a processor, implements the steps of the above-mentioned device target object detection method.

[0075] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0077] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0078] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0079] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0080] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), or electrically erasable programmable read only memory (EEPROM), for the storage of software that is read during runtime. The memory is an example of computer readable media.

[0081] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0082] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0083] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of on-device target object detection, the method comprising: The method comprises: obtaining a to-be-tested image of a device containing a target object, the target object comprising components and assembly gaps between the components on the device; segmenting the to-be-tested image to obtain a plurality of component images, each of the component images not containing the assembly gaps; based on the component images, determining a circumscribed polygonal region of a corresponding component image, wherein an extension direction of a side of the circumscribed polygonal region close to the assembly gap is parallel to a predetermined length direction of the assembly gap; based on the determined circumscribed polygonal region, determining the assembly gap in the to-be-tested image, comprising: performing a morphological closing operation on the circumscribed polygonal region corresponding to the assembly gap to be detected to obtain a first image region; and performing a difference operation on the obtained first image region and the circumscribed polygonal region before the morphological closing operation to determine the assembly gap in the to-be-tested image corresponding to the corresponding circumscribed polygonal region.

2. The method of claim 1, wherein, The circumscribed polygon is a minimum envelope rectangle of the component in the to-be-tested image.

3. The method according to any one of claims 1 to 2, characterized in that, Before the segmenting the to-be-tested image to obtain a plurality of component images, the method comprises: determining a plurality of target regions from the to-be-tested image, the target regions containing the assembly gaps; the segmenting the to-be-tested image to obtain a plurality of component images comprises: segmenting the target regions to obtain a plurality of component images.

4. The method of claim 3, wherein, The determining a plurality of target regions from the to-be-tested image comprises: determining an offset between the device in the to-be-tested image and the device in a preset template image, the preset template image containing a plurality of regions of interest, the regions of interest comprising the assembly gaps; generating regions corresponding to the regions of interest in the to-be-tested image according to the preset template image and the offset to obtain the target regions.

5. The method of claim 4, wherein, Before the segmenting the target regions to obtain a plurality of component images, the method further comprises: performing filtering processing on the target regions in a direction parallel to the assembly gap.

6. The method of claim 4, wherein, After the segmenting the target regions to obtain a plurality of component images, the method further comprises: performing direction correction on the plurality of component images to rotate the plurality of component images to a preset direction.

7. The method of claim 1, wherein, After the segmenting the to-be-tested image to obtain a plurality of component images, the method further comprises: performing a denoising operation on the component images.

8. The method of claim 7, wherein, The denoising operation on the component images comprises: hole filling on the component images; and / or performing a morphological opening operation on the component images and performing a closing operation based on the result of the opening operation.

9. The method of claim 3, wherein, If the target regions are a plurality of target regions; each target region is arranged at intervals along the length direction of the assembly gap, the determining the assembly gap in the to-be-tested image based on the determined circumscribed polygonal region comprises: in each target region, determining a corresponding assembly gap based on the determined circumscribed polygonal region; based on the plurality of assembly gaps determined in the plurality of target regions, determining the assembly gap in the to-be-tested image.

10. The method of claim 1, wherein, After the assembly gap in the to-be-tested image is obtained, the method further comprises: determining a detection result of the device according to a feature of the assembly gap.

11. A device on target object detection apparatus, characterized by, The method comprises: An image acquisition module is configured to acquire a to-be-tested image of a device containing a target object, the target object including components on the device and assembly gaps between the components; An image segmentation module is configured to segment the to-be-tested image to obtain a plurality of component images, each of the component images not containing the assembly gaps; A region determination module is configured to determine a circumscribed polygonal region of a corresponding component image based on the component image, wherein an extension direction of an edge of the circumscribed polygonal region close to the assembly gap is parallel to a predetermined length direction of the assembly gap; and A gap determination module is configured to determine the assembly gaps in the to-be-tested image based on the determined circumscribed polygonal region, including: performing a morphological closing operation on the circumscribed polygonal region corresponding to the assembly gap to be detected to obtain a first image region; and performing a difference operation on the obtained first image region and the circumscribed polygonal region before the morphological closing operation to determine the assembly gaps in the to-be-tested image corresponding to the corresponding circumscribed polygonal region.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the device target object detection method in any one of claims 1 to 10.

13. A storage medium having instructions stored therein, which, when executed on a computer, cause the computer to perform the steps of the device target object detection method in any one of claims 1 to 10.

Images