X-ray image food foreign matter detection and labeling method
Through image fusion technology, the food foreign body in X -ray images is automatically marked, which solves the problem of timely and labor -intensive labeling, and achieves high -efficiency and good quality labeling process.
Patent Information
- Application Number
- CN202411450752.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-01-21
AI Technical Summary
In the prior art, manual labeling of food foreign matter detection data in X-ray images is time-consuming and labor-intensive, and the labeling efficiency and quality are difficult to guarantee.
Through image fusion technology, multiple foreign objects are randomly selected from foreign body data sets to integrate with qualified food images, generate food foreign objects, and automatically record the labeling position to achieve automatic marking of food foreign object images.
It simplifies the food foreign matter detection process, reduces the workload of practitioners, reduces labeling costs, and improves labeling efficiency and quality.
Smart Images
Figure CN119339368A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting and labeling food foreign bodies in X-ray images. Background Art
[0002] Deep learning performs well in image recognition and feature extraction. Foreign body detection in X-ray images is an important application of deep learning in the food industry. The training of deep learning models relies on a large amount of labeled data, and data labeling can provide key training samples for the model. At present, manual labeling is the mainstream method of data labeling. However, manual labeling is often time-consuming and labor-intensive, and the labeling efficiency and quality are difficult to guarantee. In view of the time-consuming and labor-intensive manual labeling of data in food foreign body detection, in order to improve labeling efficiency and reduce repetitive workload, a labeling method for food foreign body detection in X-ray images is proposed to reduce the burden of manual labeling. Summary of the invention
[0003] In view of the shortcomings of the prior art, the present invention solves the technical problems by adopting a technical solution: a method for detecting and marking foreign matter in food using an X-ray image, comprising the following steps:
[0004] S1. Image acquisition: Use X-ray foreign body detection equipment to collect images of food samples;
[0005] S2, dataset construction: obtain the Foreign graph and construct the foreign object dataset;
[0006] S3, image fusion: randomly select multiple Foreign images from the foreign body data set, and fuse them with a single Ok image to generate food foreign body images;
[0007] S4. Automatic labeling: record the position of image fusion and save the generated image and label file, so as to achieve automatic labeling of food foreign body images.
[0008] Preferably, the X-ray sample images collected in S1 include food Ng images and food Ok images, wherein the Ng image is an X-ray image of an unqualified sample, which contains objects on the surface or inside that do not belong to the sample itself, and the Ok image is an X-ray image of a qualified sample, which does not contain foreign matter.
[0009] Preferably, the Foreign graph in S2 is mainly a foreign body of the Ng graph, and the Foreign graph is obtained by manual annotation, image segmentation and other open source image segmentation data sets.
[0010] Preferably, the manual labeling is to label all possible foreign objects contained in the image Ng of a certain type of food in a targeted manner, so as to obtain common foreign object images of this type of food.
[0011] Preferably, the image segmentation is to separate the foreign body images from the Ng food images from the historical annotation data by means of semantic segmentation, threshold segmentation and edge detection.
[0012] Preferably, the other open source image segmentation datasets include the MS COCO dataset, the PASCAL VOC 2012 dataset and the Cityscapes dataset, and the segmented Foreign images are processed to replace the foreign body images.
[0013] Preferably, before image fusion is performed in S3, the multiple Foreign images and the single Ok image need to be resized, processed and renamed respectively, and when all placement rules are met, the food foreign body image is synthesized.
[0014] Preferably, the composite image needs to determine whether the number of Foreign placements of a single image is reached. If so, the image fusion process is terminated. If not, a single Foreign image is randomly selected from the selected multiple Foreign images and image fusion is performed again.
[0015] Preferably, the food foreign body image in S4 is automatically labeled including the category C of the foreign body mark and the position P of the foreign body mark target box. The category C can be obtained according to the name of the foreign body image, and the character string before the first underscore of the file name is obtained using a regular expression according to the Foreign graph naming rules.
[0016] Preferably, the position P can be obtained according to the synthesis rule of the Ok image and the Foreign image, and the position (x, y) of the upper left corner of the Foreign image is specified according to the image sizes (w1, h1) and (w2, h2) of the Ok image and the Foreign image to ensure that it is not placed outside the Ok image, and the average grayscale of the placed area of the Ok image is calculated, and the parameter Alpha is set to adjust the fitting ratio of the Ok image and the Foreign image, and the grayscale of the foreign image in the synthesized foreign image is controlled, and the average grayscale value g1 of the placed area in the Ok image is calculated. and the minimum grayscale value g2 of the Foreign image, calculate the label position information (x0, y0, w0, h0) according to the random position (x, y) and the size information of the Ok image and the Foreign image, merge the label name and label position, return a single Foreign image label tuple L, create a TXT file with the same name based on the single Ok name, write the label tuple L to the TXT file in the form of appending, if the placement quantity N is met, the Foreign image placement ends and executes the next Ok image, otherwise the placement continues.
[0017] The beneficial effects of the present invention are as follows:
[0018] The present invention can adjust the type, quantity, grayscale and other information of foreign matter according to food production needs, which simplifies the food foreign matter detection process to a certain extent, reduces the workload of image acquisition, data annotation and model training for practitioners, and also reduces the difficulty of obtaining food foreign matter detection data sets. Compared with traditional manual annotation methods, it greatly reduces the annotation cost and improves the annotation efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a schematic diagram of an X-ray sample image in the present invention;
[0020] Figure 2 It is a schematic diagram of a foreign body picture in the present invention;
[0021] Figure 3 is a flow chart of image fusion in the present invention;
[0022] Figure 4 It is a flowchart automatically marked in the present invention;
[0023] Figure 5 It is the effect diagram of automatic annotation in the present invention. DETAILED DESCRIPTION
[0024] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0025] A method for detecting and labeling food foreign matter in an X-ray image, comprising the following steps:
[0026] S1. Image acquisition: Use X-ray foreign body detection equipment to collect images of food samples;
[0027] S2, dataset construction: obtain the Foreign graph and construct the foreign object dataset;
[0028] S3, image fusion: randomly select multiple Foreign images from the foreign body data set, and fuse them with a single Ok image to generate food foreign body images;
[0029] S4. Automatic labeling: record the position of image fusion and save the generated image and label file, so as to achieve automatic labeling of food foreign body images.
[0030] like Figure 1 As shown in the figure, the collected X-ray sample images include food Ng images and food Ok images. Among them, the Ng image is the X-ray image of the unqualified sample, and the surface or interior contains objects that do not belong to the sample itself, such as stones, metals, plastics and glass; the Ok image is the X-ray image of the qualified sample, and the sample itself does not contain foreign matter.
[0031] like Figure 2As shown in the figure, the Foreign graph is mainly the foreign matter in the Ng graph. In food foreign matter detection, it is usually necessary to mark the foreign matter area in the Ng graph. These foreign matters are generally stones, metals, plastics, glass, etc. The shape, size, and specific properties of foreign matters have certain similarities. According to the foreign matter property information, they can be classified into the following 6 categories, as shown in the following table (foreign matter classification standards):
[0032] Serial number Label Foreign matter category illustrate 0 foreign Point type Small dots, short threads, large dots 1 block Block type The plan view is an irregular polygon 2 longline Filamentous Long, thin, curved, irregular threads 3 ring Ring Concentric circles, hollow 4 screw Screws Pointed and flat screws, screws 5 shape Specific Class Paper clips, springs, utility knives, hairpins, etc.
[0033] Foreign object annotation in Ng images is mainly done manually, but manual annotation is time-consuming and laborious, and the efficiency and quality of annotation cannot be guaranteed. In this embodiment, multiple Foreign images (foreign object images) are fused with Ok images, and the position of foreign objects in Ok images is obtained through fusion rules, thereby achieving automatic annotation of the composite image, thereby replacing manual annotation of foreign objects in Ng images.
[0034] Foreign images can be obtained through manual annotation, image segmentation, and other open source image segmentation datasets. Manual annotation is to specifically annotate all possible foreign objects in the Ng images of a certain type of food, so as to obtain common foreign body images of this type of food. Image segmentation is to separate foreign body images from Ng food images in historical annotation data through semantic segmentation, threshold segmentation, and edge detection. Other open source image segmentation datasets include the MS COCO dataset, the PASCAL VOC 2012 dataset, and the Cityscapes dataset, which replace foreign body images by performing image processing on the segmented Foreign images.
[0035] like Figure 3 As shown in the figure, before image fusion, multiple Foreign images and a single Ok image need to be resized, processed, and renamed. When all placement rules are met, a food foreign body image is synthesized. The synthesized image needs to determine whether the number of Foreign placements of a single image is reached. If it is met, the image fusion process is terminated. If not, a single Foreign image is randomly selected from the selected multiple Foreign images and image fusion is performed again.
[0036] like Figure 4 As shown in the figure, the automatic labeling of food foreign body images includes the category C of the foreign body mark and the position P of the foreign body mark target frame; the category C can be obtained according to the name of the foreign body image, and the string "_" before the first underscore of the file name is obtained using a regular expression according to the Foreign image naming rules. For example: the image with the file name "screw_09-36-54-347_1.png" is labeled as "screw" in the labeling file, and the corresponding serial number in the foreign body classification standard table is 4.
[0037] The position P can be obtained according to the synthesis rules of the Ok map and the Foreign map. The specific steps are as follows:
[0038] 1) Random placement: According to the picture sizes (w1, h1) and (w2, h2) of the Ok map and the Foreign map, specify the position (x, y) of the upper left corner of the Foreign map:
[0039]
[0040] Ensure that it is not placed outside the Ok map;
[0041] 2) Placement rule: Calculate the average gray value of the placed area of the Ok map. If it is greater than 245, it indicates that it is an area with a relatively high gray value and is relatively white in the Ok map. The food foreign object detection effect in this area is poor. Return to the first step;
[0042] 3) Fitting ratio: Set the parameter Alpha to adjust the fitting ratio of the Ok map and the Foreign map:
[0043] alpha ∈ (-1, 1)
[0044] F_pic = mask * (1 - alpha) + ok * alpha
[0045] Control the gray value of the foreign object image in the synthesized foreign object image;
[0046] 4) Inspection rule: Calculate the average gray value g1 of the placed area in the Ok map and the minimum gray value g2 of the Foreign map. If g1 < g2 + g (debug value), it indicates that the gray value of the processed Foreign map is higher than that of the Ok map, and the significance is not great. Return to the first step;
[0047] 5) Calculate the position: Calculate the label position information (x0, y0, w0, h0) according to the random position (x, y) and the size information of the Ok map and the Foreign map:
[0048]
[0049] 6) Return the label: Combine the label name and the label position, and return the single Foreign map label tuple L:
[0050] L = (C, x0, y0, w0, h0)
[0051] 7) Save the label: Create a TXT file with the same name based on the name of a single Ok map, and write the label tuple L to the TXT file in an appended form. If the placement quantity N is satisfied, the placement of the Foreign map ends and the next Ok map is executed. Otherwise, return to the first step and continue the placement.
[0052] Convert the annotation file from TXT to XML or JSON and view it using the corresponding annotation tools, such as Figure 5 As shown in the figure, the automatic labeling effect of different food foreign body images generated after the same Ok image data is augmented. The software is the open source labeling tool Labelimg.
[0053] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.
Claims
1. A method for detecting and labeling food foreign matter in X-ray images, characterized in that: The following steps are involved: S1. Image acquisition: Use X-ray foreign body detection equipment to collect images of food samples; S2, dataset construction: obtain the Foreign graph and construct the foreign object dataset; S3, image fusion: randomly select multiple Foreign images from the foreign body data set, and fuse them with a single Ok image to generate food foreign body images; S4. Automatic labeling: record the position of image fusion and save the generated image and label file, so as to achieve automatic labeling of food foreign body images.
2. The method for detecting and labeling food foreign matter in X-ray images according to claim 1, characterized in that: The X-ray sample images collected in S1 include food Ng images and food Ok images, wherein the Ng image is an X-ray image of an unqualified sample, which contains objects on the surface or inside that do not belong to the sample itself, and the Ok image is an X-ray image of a qualified sample, which does not contain foreign matter.
3. The method for detecting and labeling food foreign matter in X-ray images according to claim 2, characterized in that: The Foreign graph in S2 is mainly a foreign object of the Ng graph. The Foreign graph is obtained by manual annotation, image segmentation and other open source image segmentation datasets.
4. The method for detecting and labeling food foreign matter in X-ray images according to claim 3, characterized in that: The manual labeling is to label all possible foreign objects contained in a certain type of food Ng image in a targeted manner, so as to obtain common foreign object images of this type of food.
5. The method for detecting and labeling food foreign matter in X-ray images according to claim 3, characterized in that: The image segmentation is to separate foreign body images from Ng food images from historical annotation data by means of semantic segmentation, threshold segmentation and edge detection.
6. The method for detecting and labeling food foreign matter in X-ray images according to claim 5, characterized in that: The other open source image segmentation datasets include the MS COCO dataset, the PASCAL VOC 2012 dataset, and the Cityscapes dataset, which replace the foreign body images by performing image processing on the segmented Foreign images.
7. The method for detecting and labeling food foreign matter in X-ray images according to claim 1, characterized in that: In S3, before image fusion, multiple Foreign images and a single Ok image need to be resized, processed, and renamed. When all placement rules are met, a food foreign body image is synthesized.
8. The method for detecting and labeling food foreign matter in X-ray images according to claim 7, characterized in that: The composite image needs to determine whether the number of Foreign placements of a single image is reached. If so, the image fusion process is terminated. If not, a single Foreign image is randomly selected from the selected multiple Foreign images and image fusion is performed again.
9. The method for detecting and labeling food foreign matter in X-ray images according to claim 1, characterized in that: The automatic labeling of the food foreign body image in S4 includes the category C of the foreign body mark and the position P of the foreign body mark target box. The category C can be obtained according to the name of the foreign body image, and the character string before the first underscore of the file name is obtained using a regular expression according to the Foreign graph naming rule.
10. The method for detecting and labeling food foreign matter in X-ray images according to claim 9, characterized in that: The position P can be obtained according to the synthesis rule of the Ok image and the Foreign image. According to the image sizes (w1, h1) and (w2, h2) of the Ok image and the Foreign image, the position (x, y) of the upper left corner of the Foreign image is specified to ensure that it is not placed outside the Ok image. The average grayscale of the placed area of the Ok image is calculated, and the parameter Alpha is set to adjust the fitting ratio of the Ok image and the Foreign image to control the grayscale of the foreign image in the synthesized foreign image. The average grayscale value g1 and F of the placed area in the Ok image are calculated. The minimum grayscale value g2 of the oreign image is calculated, and the label position information (x0, y0, w0, h0) is calculated according to the random position (x, y) and the size information of the Ok image and the Foreign image. The label name and label position are merged, and a single Foreign image label tuple L is returned. A TXT file with the same name is created based on the single Ok name, and the label tuple L is written to the TXT file in an appended form. If the placement quantity N is met, the Foreign image placement ends and the next Ok image is executed, otherwise the placement continues.