Target detection methods, apparatus, electronic devices and computer-readable storage media

Abstract

This application relates to the field of visual algorithm technology and provides a target detection method, apparatus, electronic device, and computer-readable storage medium, including: acquiring a first image and a second image; detecting target regions containing the same target in the first image and the second image; determining the rate of change of pixels in the target region; and determining the target detection result based on the rate of change. This application processes acquired product images to determine the rate of change of the detected target regions in the two images, and determines the detection result of the product in the images based on the rate of change of the target regions. This achieves the purpose of batch detecting product performance using acquired images, improving the efficiency of product performance detection compared to existing methods for software testing of sampled products.

Description

Technical Field

[0001] This application belongs to the field of visual algorithm technology, and in particular relates to object detection methods, devices, electronic devices and computer-readable storage media. Background Technology

[0002] With the continuous development of the manufacturing industry, in order to ensure the production quality of automated production lines, manual monitoring is usually set up in the product production process. By monitoring changes in equipment or instruments with the human eye, products that may have abnormalities can be identified in a timely manner.

[0003] Currently, in order to solve the problems of high cost and low accuracy in the process of manual monitoring of production lines, artificial intelligence technology has been introduced into the field of visual algorithms, and an automated testing system has been proposed. This system not only uses the automated monitoring system to perform image recognition on the monitored images to identify products with abnormal appearance, but also conducts random inspections of products on the production line and performs functional tests on the inspected products to achieve the purpose of testing product performance.

[0004] However, existing methods for testing product functionality require one-to-one software testing of each sampled product, resulting in low testing efficiency. Summary of the Invention

[0005] This application provides a target detection method, apparatus, electronic device, and computer-readable storage medium, which achieve the goal of improving product detection efficiency.

[0006] In a first aspect, embodiments of this application provide a target detection method, including:

[0007] Obtain the first image and the second image;

[0008] Detect target regions in the first image and the second image that contain the same target;

[0009] Determine the rate of change of pixels in the target region;

[0010] The target detection result is determined based on the rate of change.

[0011] For example, multiple image acquisition devices are set up on an automated production line to simultaneously capture images of multiple products. Two consecutive images are used as the first and second images, respectively. The production line is then controlled to play video on the products, with the video continuously switching between different display screens. It should be understood that the time interval between capturing the first and second images is greater than the time interval between the product display screen transitions. Therefore, it is possible to identify target areas belonging to the same product in the two consecutive images, and to confirm whether the product corresponding to the target area has a display screen transition malfunction based on the calculated pixel change rate of the target area.

[0012] As can be seen from the above embodiments, by processing the product images collected from the production line, determining the rate of change of the target area detected in the two images, and identifying the products with display problems in the images based on the rate of change of the target area, the purpose of batch testing the display performance of products using the collected images is achieved, thereby improving the efficiency of product performance testing.

[0013] In one possible implementation of the first aspect, detecting target regions containing the same target in the first image and the second image includes:

[0014] Identify at least one first detection box contained in the first image, and identify at least one second detection box contained in the second image;

[0015] At least one group of detection boxes is obtained, wherein each group of detection boxes includes a first detection box and a second detection box;

[0016] Determine the matching degree of each of the detection box groups;

[0017] The target detection box group is determined based on the matching degree;

[0018] Determine the target region corresponding to the target detection box group.

[0019] It should be understood that the acquired first and second images contain multiple products. For example, a detection bounding box for each product in the two images can be detected using a target detection model. Since the two images capture multiple products at the same location on the production line, the detection bounding boxes for the multiple products in the two images are all from the same batch of products. Furthermore, based on the matching degree of the detection bounding boxes in the two images, a group of target detection bounding boxes belonging to the same product can be identified, thereby determining the target region belonging to the same product in the two images.

[0020] As can be seen from the above embodiments, by identifying the detection boxes corresponding to all products contained in the first image and the second image, calculating the matching degree of the detection box group in the two images, and determining the target area belonging to the same product in the two images based on the matching degree of the detection box group, the target area corresponding to each product contained in the two images is detected, thereby achieving the purpose of batch testing the product display performance on the production line and improving the testing efficiency of product performance.

[0021] In one possible implementation of the first aspect, determining the matching degree of each of the detection box groups includes:

[0022] For each group of detection boxes, the overlapping area of ​​the first detection box and the second detection box is determined;

[0023] The ratio of the overlapping region to the first detection frame is taken as the first degree of overlap, and the ratio of the overlapping region to the second detection frame is taken as the second degree of overlap.

[0024] The matching degree of the detection box group is determined based on the first overlap degree and the second overlap degree.

[0025] It should be understood that after identifying the detection boxes of all products contained in the two images, the first detection box contained in the first image and the second detection box contained in the second image are combined, and the target areas belonging to the same product are identified by comparing the matching degree of the combined detection box group.

[0026] As can be seen from the above embodiments, in the process of calculating the matching degree of the first detection box and the second detection box contained in the detection box group, the matching degree calculated based on the overlapping area of ​​the two detection boxes provided in this application, compared with the intersection-union ratio calculation method used by the existing target recognition algorithm, can accurately identify whether the two detection boxes belong to the same target when there is occlusion or missing detection boxes, thereby improving the accuracy of detecting target areas belonging to the same product.

[0027] In one possible implementation of the first aspect, determining the target region corresponding to the target detection box group includes:

[0028] For the target detection box group, a first target overlapping region included in the first detection box and a second target overlapping region included in the second detection box are determined, wherein the first target overlapping region and the second target overlapping region are located at the same position.

[0029] The overlapping region of the first target and the overlapping region of the second target are taken as the target region corresponding to the target detection box group.

[0030] It should be understood that when it is confirmed that the first detection box and the second detection box contained in the target detection box group belong to the same target being detected, the overlapping part of the two detection boxes is taken as the target area to be detected for pixel change rate. That is, the target area includes the first target overlapping area of ​​the first detection box and the second target overlapping area of ​​the second detection box.

[0031] In one possible implementation of the first aspect, determining the rate of change of pixels in the target region includes:

[0032] Obtain the first pixel matrix corresponding to the overlapping region of the first target;

[0033] Obtain the second pixel matrix corresponding to the overlapping region of the second target;

[0034] The rate of change of pixels in the target region is determined based on the difference between the first pixel matrix and the second pixel matrix.

[0035] It should be understood that after identifying the target area belonging to the same product under test, the video played by the product will continuously switch the display screen. Therefore, the display screen can be identified based on the rate of change of pixels belonging to the target area of ​​the same product under test in two consecutive images.

[0036] As can be seen from the above embodiments, by calculating the pixel difference between the first target overlapping area of ​​the first detection frame and the second target overlapping area of ​​the second detection frame contained in the target area, it is possible to confirm whether the product corresponding to the current target area has a fault of switching display images, thus achieving the purpose of product display performance detection based on the collected images.

[0037] In one possible implementation of the first aspect, determining the rate of change of pixels in the target region based on the difference between the first pixel matrix and the second pixel matrix includes:

[0038] Based on the difference between the first pixel matrix and the second pixel matrix, the change value of the pixels contained in the target region and the number of pixels contained in the target region are obtained;

[0039] Pixels whose change value is greater than a preset pixel threshold are taken as target pixels;

[0040] The ratio of the number of target pixels to the number of pixels contained in the target region is used as the rate of change of pixels in the target region.

[0041] As can be seen from the above embodiments, pixels with a change value greater than a preset pixel threshold are taken as target pixels, and the change rate of the target area is determined based on the proportion of the target pixels in the target area. This eliminates the interference of environmental noise on the displayed image and improves the accuracy of the calculated change rate of pixels in the target area.

[0042] In one possible implementation of the first aspect, identifying at least one first detection box contained in the first image includes:

[0043] The first image is identified by an object detection model, which is trained on a labeled training set. The labeled training set contains at least one labeled sample image, and the rectangular labeled box of each labeled sample image is a labeled box that is greater than or equal to a preset detection box.

[0044] As can be seen from the above embodiments, the larger the detection box corresponding to the product in the image, the higher the accuracy of product performance identification using the method of this application. Therefore, in the process of training the target detection model, the annotation boxes of the tested products in the sample images of the training set can be set to be greater than or equal to the preset detection boxes. That is, the training set is optimized to induce the model to only detect targets in the image whose annotation boxes are larger than the preset detection boxes, so that the detected targets are all the parts of the product that are closer to the shooting distance, thereby ensuring the accuracy of the product performance detection results of this application.

[0045] Secondly, embodiments of this application provide a target matching device, comprising:

[0046] The acquisition module is used to acquire the first image and the second image;

[0047] The detection module is used to detect target regions in the first image and the second image that contain the same target;

[0048] The first determining module is used to determine the rate of change of pixels in the target region;

[0049] The second determining module is used to determine the target detection result based on the rate of change.

[0050] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the target detection method as described in any one of the first aspects above.

[0051] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the target detection method as described in any one of the first aspects above.

[0052] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the target detection method described in any one of the first aspects.

[0053] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the target detection method provided in the embodiments of this application. Figure 1 ;

[0056] Figure 2 This is a schematic image provided in an embodiment of this application. Figure 1 ;

[0057] Figure 3 This is a schematic image provided in an embodiment of this application. Figure 2 ;

[0058] Figure 4 This is a schematic image provided in an embodiment of this application. Figure 3 ;

[0059] Figure 5 This is a flowchart illustrating the target detection method provided in the embodiments of this application. Figure 2 ;

[0060] Figure 6 This is a schematic diagram of the target detection device provided in the embodiments of this application;

[0061] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0062] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0063] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0064] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0065] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0066] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0067] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0068] With the continuous development of the manufacturing industry, in order to solve the problems of high cost and low accuracy in manual monitoring of production lines, artificial intelligence technology has been introduced into the field of visual algorithms, proposing an automated testing system. This automated monitoring system uses image recognition to identify products with abnormal appearance. For performance testing of products on the production line, products are typically sampled and subjected to one-to-one software testing. The performance of the products produced on the current production line is then assessed based on the results of the sampled products' performance tests. However, existing methods for testing product performance using testing software suffer from low detection efficiency.

[0069] To address the aforementioned technical issues, this application processes the acquired product images to determine the rate of change of the target area detected in the two images taken before and after the capture. Based on the rate of change of the target area, the detection result of the product in the image is determined, thereby achieving the goal of batch testing product performance using the acquired images. Compared with existing methods for software testing of sampled products, this method improves the efficiency of product performance testing.

[0070] See Figure 1 This is a schematic flowchart of the target detection method provided in an embodiment of this application. As an example and not a limitation, the method may include the following steps:

[0071] S101: Obtain the first image and the second image.

[0072] In this embodiment, an automated monitoring system consisting of multiple image acquisition devices is deployed on the product production line. The automated monitoring system has multiple image acquisition devices that take pictures of multiple products at preset time intervals.

[0073] In this embodiment, for multiple images captured by the same image acquisition device within the same production area, two consecutive images are designated as the first image and the second image. It should be understood that the first image and the second image capture the same production line area; that is, the first image and the second image are images of the same batch of products at different times.

[0074] In this embodiment, a test video is played on the product line. Specifically, the test video is a video with continuously switching display screens. It should be understood that the time interval between capturing the first image and the second image is set to be greater than the time interval between switching product display screens.

[0075] As an example, the automated monitoring system set in this embodiment is used to detect the display function of production equipment on the production line. Figure 2 This is a schematic image provided in an embodiment of this application. Figure 1 .like Figure 2 As shown, an automated monitoring system is configured with an image acquisition device to capture images of multiple devices on the production line. Specifically, after the image acquisition device is activated, it captures images at 1-second intervals. The image captured in the first second is used as the first image, and the image captured in the second second is used as the second image. That is, the first and second images contain the display interfaces of multiple devices at different times.

[0076] S102: Detect target regions in the first and second images that contain the same target.

[0077] In this embodiment, a preset image contained in the acquired first and second images is identified. For example, a target detection model based on the YOLOv5 algorithm is used to identify the preset image contained in the first image. Specifically, in this embodiment, the preset image is set to the device display interface.

[0078] In this embodiment, the process of obtaining the target detection model is as follows: First, multiple product images taken on the production line are collected as a training set. Second, data labels are created to annotate the images in the training set. Then, the annotated training set is used to train the single-stage model, so that the trained target detection model can identify and annotate the target annotation boxes of the device display interface in the image.

[0079] Figure 3 This is a schematic image provided in an embodiment of this application. Figure 2 .like Figure 3 As shown, a trained object detection model is used to identify the device display interface in the first image and the second image, and at least one detection box in the first image and at least one detection box in the second image are identified. It should be understood that each detection box in the first image and the second image corresponds to a device display interface.

[0080] In one possible implementation, after identifying at least one detection box contained in the first image and at least one detection box contained in the second image, the at least one detection box contained in the first image is designated as the first detection box, and the at least one detection box contained in the second image is designated as the second detection box. Then, the Cross-Union Comparison (CUC) algorithm is used to process all the first detection boxes contained in the first image and the second detection boxes contained in the second image to identify detection boxes belonging to the same target in both the first and second images. It should be understood that the implementation process of using the CUC algorithm to identify detection boxes in target detection tasks is existing technology and will not be elaborated upon here.

[0081] As an example, in Figure 3 After identifying at least one detection box contained in the first image and at least one detection box contained in the second image, as follows: Figure 3 As shown, Figure 3 The first image contains first detection boxes A1, B1, and C1, and the second image contains first detection boxes A2, B2, and C2. The cross-union algorithm is applied to all first detection boxes in the first image and all second detection boxes in the second image to identify that first detection boxes A1 and second detection boxes A2 contain the same target A, first detection boxes B1 and second detection boxes B2 contain the same target B, and first detection boxes C1 and second detection boxes C2 contain the same target C.

[0082] In this embodiment, for a first detection box and a second detection box belonging to the same target, the overlapping portion of the regions contained in the first detection box and the second detection box is taken as the target region corresponding to the target of the current detection box. It should be understood that this step identifies the target regions corresponding to all targets identified in the first image and the second image.

[0083] For example, such as Figure 3 As shown, after identifying that the first detection box A1 and the second detection box A2 contain the same target A, the overlapping portion of the regions contained in the first detection box A1 and the second detection box A2 is taken as the target region of target A. It should be understood that the intersection-union algorithm will identify the target regions corresponding to target A, target B, and target C respectively.

[0084] S103: Determine the rate of change of pixels in the target region.

[0085] In this embodiment, it should be understood that after identifying the target area of ​​the same product in the two images in S102, by setting the screen of the device under test to switch, the rate of change of pixels of the same target area in the two images can be calculated to identify whether the device corresponding to the target has a screen switching fault.

[0086] In one possible implementation, for each target region, the overlapping region in the first detection frame is taken as the first target overlapping region, and the overlapping region in the second detection frame is taken as the second target overlapping region. In this embodiment, a first pixel matrix corresponding to the first target overlapping region is obtained; a second pixel matrix corresponding to the second target overlapping region is obtained; and the rate of change of pixels in the target region is determined based on the difference between the first pixel matrix and the second pixel matrix.

[0087] As an example, for target A, the portion of the first detection box A1 that overlaps with the second detection box A2 is defined as the first target overlapping region a1, and the portion of the second detection box A2 that overlaps with the first detection box A1 is defined as the second target overlapping region a2. Further, a first pixel matrix corresponding to the first target overlapping region a1 is obtained based on the grayscale values ​​of all pixels within the first target overlapping region a1, and a second pixel matrix corresponding to the second target overlapping region a2 is obtained based on the grayscale values ​​of all pixels within the second target overlapping region a2. Therefore, the pixel change rate of target A in the two images can be determined by calculating the first pixel matrix and the second pixel matrix.

[0088] In one possible embodiment, the specific steps for determining the pixel change rate of target A in two consecutive images by calculating the first pixel matrix and the second pixel matrix are as follows: based on the difference between the first pixel matrix and the second pixel matrix, obtain the change value of the pixels contained in the target region and the number of pixels contained in the target region; take the pixels with change values ​​greater than a preset pixel threshold as target pixels; and take the ratio of the number of target pixels to the number of pixels contained in the target region as the pixel change rate in the target region.

[0089] In this embodiment, in order to eliminate the interference of environmental noise on the displayed image, a preset pixel threshold is set according to the environmental noise, and pixels whose difference between the first pixel matrix and the second pixel matrix is ​​greater than the preset pixel threshold are taken as target pixels with pixel changes. Then, the ratio of the number of target pixels to the number of pixels in the overlapping area is taken as the rate of change of pixels in the target area.

[0090] As an example, after obtaining the first pixel matrix corresponding to the first target overlapping region a1 and the second pixel matrix corresponding to the second target overlapping region a2, the difference between the first pixel matrix and the second pixel matrix is ​​calculated to obtain the pixel difference matrix. Further, the number of target pixels with pixel values ​​greater than a preset pixel threshold in the pixel difference matrix is ​​determined to be 20, and the number of pixels contained in the first target overlapping region a1 and the second target overlapping region a2 is 100. Therefore, the final calculated pixel change rate in the target region is 80%.

[0091] S104: Determine the target detection result based on the rate of change.

[0092] In this embodiment, the presence of a change in the image displayed in the corresponding target area is determined based on the rate of change of pixels in the target area. In one possible implementation, when the rate of change of pixels in the target area is less than a preset minimum rate of change, the target detection result corresponding to the current target area is determined to be negative, indicating a display screen switching fault in the device corresponding to the current target area. In this embodiment, a preset minimum rate of change is set based on the switched screen. When the rate of change of pixels in the target area is less than the preset minimum rate of change, it can be determined that the rate of change in the current target area is low, meaning that a screen switching fault exists in the image displayed for the target corresponding to the current target area.

[0093] As an example, a preset minimum change rate is set to 20%. For instance, after obtaining the first pixel matrix corresponding to the first target overlapping region a1 and the second pixel matrix corresponding to the second target overlapping region a2, and calculating the change rate of pixels in the target region to 80%, the change rate of pixels in the target region to 80% is greater than the preset minimum change rate of 20%. Therefore, the target detection result corresponding to target A is determined to be a successful test.

[0094] For example, after calculating the pixel change rate of 15% in the target region corresponding to target B, the pixel change rate of 15% in the target region corresponding to target B is less than the preset minimum change rate of 20%. Therefore, the target detection result corresponding to target B is determined to be faulty.

[0095] In one possible implementation, when the above steps confirm that a faulty target detection result appears in the first image and the second image, the first detection box and the second detection box corresponding to the faulty target area are set as highlighted annotation boxes, and the first image and the second image after highlighting are transmitted to the background to promptly investigate the device with the screen switching fault.

[0096] As can be seen from the above embodiments, based on the obtained first image and second image, target regions containing the same target in the first and second images are detected, and the rate of change of pixels in the target region is determined. Finally, the target detection result is determined based on the rate of change. This application processes the acquired product images to determine the rate of change of the detected target regions in the two images, and determines the detection result of the product in the images based on the rate of change of the target regions. This achieves the purpose of batch detection of product performance using acquired images. Compared with existing methods for software testing of sampled products to detect product performance, it can eliminate the interference of environmental noise, improve the accuracy of calculating the frame difference results corresponding to the target regions, and make the target detection results more accurate, thereby improving the efficiency of product performance detection.

[0097] It should be understood that the display area of ​​the device under test on the production line in the first and second images conforms to the characteristic of near objects appearing larger and far objects appearing smaller. Figure 4 This is a schematic image provided in an embodiment of this application. Figure 3 .like Figure 4 As shown, when the distance between targets A, B, and C and the image acquisition device increases from near to far, the bounding boxes corresponding to targets A, B, and C in the first and second images show that the closer targets are larger and the farther targets are smaller, and the targets in front may occlude the targets behind. At this time, according to... Figure 1 The cross-union algorithm proposed in the embodiment can identify target regions belonging to the same target in the first image and the second image.

[0098] However, when the target captured in the first or second image is obscured or missing, using Figure 1 The intersection-union algorithm proposed in the example may produce erroneous results where the identified target regions do not belong to the same target. For example... Figure 4 As shown, the first image acquired includes first detection boxes A1, B1, and C1, while the second image acquired includes first detection boxes A2 and C2. That is, no detection box for target B was acquired in the second image. At this point, according to... Figure 1In the embodiment of the intersection-union algorithm, since there is no target B occlusion of target A in the second image, the first detection box B1 and the first detection box B2 have no overlapping area. That is, the area of ​​the overlapping area of ​​the first detection box B1 and the second detection box A2 is greater than the area of ​​the overlapping area of ​​the first detection box B1 and the first detection box B2. Therefore, the intersection-union ratio of the first detection box B1 and the first detection box A2 is greater than that of the first detection box B1 and the first detection box B2. This leads to the identification that the first detection box B1 and the first detection box A2 belong to the same target, that is, the identified target area is incorrect, which in turn leads to the inaccuracy of the subsequent target detection results.

[0099] To ensure that the identified target regions belong to the same target, in one possible implementation, after obtaining the first image and the second image, the process for determining the target region provided in this embodiment may specifically include:

[0100] S501: Identify at least one first detection box contained in the first image, and identify at least one second detection box contained in the second image.

[0101] S502: Obtain at least one group of detection boxes, wherein each group of detection boxes contains a first detection box and a second detection box.

[0102] The methods and effects of S501 to S502 Figure 1 The method and effect implemented in S102 of the embodiment are the same, and will not be described again here.

[0103] S503: Determine the matching degree of each detection box group.

[0104] In one possible implementation, two detection boxes belonging to the same target can be determined by calculating the matching degree of each detection box group. In this embodiment, for each detection box group, the overlapping area of ​​the first detection box and the second detection box is determined; the ratio of the overlapping area to the first detection box is taken as the first overlap degree, and the ratio of the overlapping area to the second detection box is taken as the second overlap degree; the matching degree of the detection box group is determined based on the first overlap degree and the second overlap degree.

[0105] For example, the IOEA (intersection over each area) method is used to calculate the matching degree of each detection box group. Specifically, the calculation formula of the IOEA method is shown in (1):

[0106]

[0107] Where intersection is the size of the overlapping area between the first detection box in the first image and the second detection box in the second image, area1 is the size of the area of ​​the first detection box, and area2 is the size of the area of ​​the second detection box.

[0108] Specifically, The coordinates of the top-left corner of the first detection box in the first image are: The coordinates are the lower right corner of the first detection box in the first image. The coordinates are the top-left corner coordinates of the second detection box in the second image. Let (x0, y0) be the coordinates of the lower right corner of the second detection box in the second image. Let (x1, y1) be the coordinates of the upper left corner of the overlapping area region intersection, and (x1, y1) be the coordinates of the lower right corner of the overlapping area region intersection. Thus, the calculation formulas for intersection, area1, and area1 are shown in (2), (3), and (4), respectively:

[0109] intersection=(x1-x0)(y1-y0) (2)

[0110]

[0111]

[0112] In this embodiment, when the target being tested is occluded or missing in the first or second image, the IOEA method proposed in this application can accurately detect the target regions belonging to the same target.

[0113] For example, such as Figure 4 As shown, the second detection box A2 and the second detection box A2 have a positional overlap relationship, and the first detection box B1 and the second detection box A2 also have a positional overlap relationship. The matching degree between the first detection box A1 and the second detection box A2, and the matching degree between the first detection box B1 and the second detection box A2 are calculated using the IOEA method.

[0114] S504: Determine the target detection box group based on the matching degree.

[0115] In this embodiment, for each first detection box, after obtaining the matching degree of all detection box groups composed of the current first detection box, the detection box group with the highest matching degree is taken as the target detection box group of the current first detection box. Therefore, the second detection boxes included in the target detection box group are detection boxes containing the same target as the first detection box.

[0116] As an example, according to Figure 4 Based on the positional relationship of the example, it is determined that IOEA(A1, A2) is greater than IOEA(B1, A2). Therefore, the first detection box A1 and the second detection box A2 are taken as the target detection box group.

[0117] In one possible implementation, after determining the matching value corresponding to each detection box group, and before determining the target detection box group based on the matching degree, the matching value corresponding to each detection box group is filtered according to a preset detection threshold. Detection box groups with matching values ​​greater than the preset detection threshold are selected, and then the target detection box group is determined using the matching degree.

[0118] As an example, a preset detection threshold is set to 20%. For instance, both IOEA(A1, A2) and IOEA(B1, A2) are greater than the preset detection threshold. In this case, IOEA(A1, A2) is greater than IOEA(B1, A2), meaning that the first detection box A1 and the second detection box A2 are considered as a target detection box group.

[0119] S505: Determine the target region corresponding to the target detection box group.

[0120] In this embodiment, for a target detection box group, a first target overlapping region included in the first detection box and a second target overlapping region included in the second detection box are determined, wherein the first target overlapping region and the second target overlapping region are located at the same position; the first target overlapping region and the second target overlapping region are taken as the target region corresponding to the target detection box group.

[0121] For example, such as Figure 4 As shown, the first detection box A1 and the second detection box A2 are used as a target detection box group. The area in the first detection box A1 that overlaps with the second detection box A2 is taken as the first target overlapping area, and the area in the second detection box A2 that overlaps with the first detection box A1 is taken as the second target overlapping area. The first target overlapping area and the second target overlapping area are taken as the target areas corresponding to the first detection box A1 and the second detection box A2.

[0122] The target detection method provided in this embodiment calculates the matching degree between the first and second detection boxes in the detection box group. Compared with the intersection-union ratio calculation method used by existing target recognition algorithms, the matching degree calculated based on the overlapping area of ​​the two detection boxes can accurately identify whether the two detection boxes belong to the same target when there is occlusion or missing detection boxes, thereby improving the accuracy of detecting target areas belonging to the same product.

[0123] In one possible implementation, at least one first detection box is identified in the first image based on an object detection model, wherein the object detection model is trained on a labeled training set, the labeled training set containing at least one labeled sample image, and the rectangular label box of each labeled sample image is a label box that is greater than or equal to a preset detection box.

[0124] In this embodiment, a target detection model based on a single-stage target detection algorithm can be used to detect the bounding boxes corresponding to multiple products contained in the image of the production line. It should be understood that the closer the products are in the captured image, the larger the corresponding bounding box of the product in the image. When the bounding box is larger, the accuracy of product performance identification using the method of this application is higher.

[0125] In one possible implementation, during the acquisition of sample images, the shooting angle of the image acquisition device can be set to simulate the actual scene of photographing products on a production line. A training set is obtained by acquiring sample images with slight tilt. After labeling the training set, YOLOv5s is selected as the object detection algorithm for transfer learning on the labeled training set. An adaptive learning rate is used for training, with the batch size set to 2, the initial learning rate to 0.01, the momentum to 0.937, and the weight decay to 0.0005. The model obtained after training is used as the object detection model in this embodiment.

[0126] As can be seen from the above embodiments, in the process of training the target detection model, the bounding boxes of the products to be tested in the sample images of the training set can be set to be greater than or equal to the bounding boxes of the preset detection boxes. That is, the training set is optimized to induce the model to only detect targets in the image whose bounding boxes are greater than the preset detection boxes, so that the detected targets are all the products that are close to the shooting distance, thus ensuring the accuracy of the product performance results of this application.

[0127] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0128] Corresponding to the target detection method described in the above embodiments, Figure 6 This is a structural block diagram of the target detection device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0129] Reference Figure 6 The target detection device includes: an acquisition module 601, a detection module 602, a first determination module 603, and a second determination module 604.

[0130] The module 601 is used to obtain the first image and the second image.

[0131] The detection module 602 is used to detect target regions in the first image and the second image that contain the same target.

[0132] The first determining module 603 is used to determine the rate of change of pixels in the target region.

[0133] The second determining module 604 is used to determine the target detection result based on the rate of change.

[0134] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0135] in addition, Figure 6 The target detection device shown can be a software unit, hardware unit, or a combination of software and hardware built into an existing electronic device, or it can be integrated into the electronic device as a separate component, or it can exist as a standalone electronic device.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 7 As shown, the electronic device of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described target detection method embodiments.

[0138] The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 7This is merely an example of an electronic device and does not constitute a limitation on electronic devices. It may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0139] The processor 70 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] In some embodiments, the memory 71 may be an internal storage unit of the electronic device, such as a hard disk or memory. In other embodiments, the memory 71 may be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 71 may include both internal and external storage units of the electronic device. The memory 71 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0141] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0142] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0144] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target detection method, characterized in that, include: Obtain the first image and the second image; Detect target regions in the first image and the second image that contain the same target; Determine the rate of change of pixels in the target region; The target detection result is determined based on the rate of change. The detection of target regions containing the same target in the first image and the second image includes: Identify at least one first detection box contained in the first image, and identify at least one second detection box contained in the second image; At least one group of detection boxes is obtained, wherein each group of detection boxes includes a first detection box and a second detection box; Determine the matching degree of each of the detection box groups; The target detection box group is determined based on the matching degree; Determine the target region corresponding to the target detection box group; Determining the matching degree of each of the detection box groups includes: For each group of detection boxes, the overlapping area of ​​the first detection box and the second detection box is determined; The ratio of the overlapping region to the first detection frame is taken as the first degree of overlap, and the ratio of the overlapping region to the second detection frame is taken as the second degree of overlap. The matching degree of the detection box group is determined based on the first overlap degree and the second overlap degree; The matching degree of each detection box group is calculated using the IOEA method, and the calculation formula of the IOEA method is shown in (1): (1) Where intersection is the size of the overlapping area between the first detection box in the first image and the second detection box in the second image, area1 is the size of the area of ​​the first detection box, and area2 is the size of the area of ​​the second detection box.

2. The method according to claim 1, characterized in that, Determining the target region corresponding to the target detection box group includes: For the target detection box group, a first target overlapping region included in the first detection box and a second target overlapping region included in the second detection box are determined, wherein the first target overlapping region and the second target overlapping region are located at the same position. The overlapping region of the first target and the overlapping region of the second target are taken as the target region corresponding to the target detection box group.

3. The method according to claim 2, characterized in that, Determining the rate of change of pixels in the target region includes: Obtain the first pixel matrix corresponding to the overlapping region of the first target; Obtain the second pixel matrix corresponding to the overlapping region of the second target; The rate of change of pixels in the target region is determined based on the difference between the first pixel matrix and the second pixel matrix.

4. The method according to claim 3, characterized in that, Determining the rate of change of pixels in the target region based on the difference between the first pixel matrix and the second pixel matrix includes: Based on the difference between the first pixel matrix and the second pixel matrix, the change value of the pixels contained in the target region and the number of pixels contained in the target region are obtained; Pixels whose change value is greater than a preset pixel threshold are taken as target pixels; The ratio of the number of target pixels to the number of pixels contained in the target region is used as the rate of change of pixels in the target region.

5. The method according to claim 1, characterized in that, The identification of at least one first detection box contained in the first image includes: The first image is identified by an object detection model, which is trained on a labeled training set. The labeled training set contains at least one labeled sample image, and the rectangular labeled box of each labeled sample image is a labeled box that is greater than or equal to a preset detection box.

6. A target matching device, characterized in that, include: The acquisition module is used to acquire the first image and the second image; The detection module is used to detect target regions in the first image and the second image that contain the same target; The first determining module is used to determine the rate of change of pixels in the target region; The second determining module is used to determine the target detection result based on the rate of change; The detection of target regions containing the same target in the first image and the second image includes: Identify at least one first detection box contained in the first image, and identify at least one second detection box contained in the second image; At least one group of detection boxes is obtained, wherein each group of detection boxes includes a first detection box and a second detection box; Determine the matching degree of each of the detection box groups; The target detection box group is determined based on the matching degree; Determine the target region corresponding to the target detection box group; Determining the matching degree of each of the detection box groups includes: For each group of detection boxes, the overlapping area of ​​the first detection box and the second detection box is determined; The ratio of the overlapping region to the first detection frame is taken as the first degree of overlap, and the ratio of the overlapping region to the second detection frame is taken as the second degree of overlap. The matching degree of the detection box group is determined based on the first overlap degree and the second overlap degree; The matching degree of each detection box group is calculated using the IOEA method, and the calculation formula of the IOEA method is shown in (1): (1) Where intersection is the size of the overlapping area between the first detection box in the first image and the second detection box in the second image, area1 is the size of the area of ​​the first detection box, and area2 is the size of the area of ​​the second detection box.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the target detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the target detection method as described in any one of claims 1 to 5.

Images