Defect detection method, device, equipment, medium and program product

Abstract

The invention relates to a defect detection method, device and equipment, a medium and a program product. The defect detection method comprises the following steps: acquiring an acquisition image of a target cigarette product, and extracting image feature data of the acquisition image; the acquired image comprises an acquisition background and an acquisition target, and the acquisition target comprises a target cigarette product; obtaining reference distribution data corresponding to the collection target; the reference distribution data comprises position distribution data of at least one historical collection target in the corresponding historical collection image; the collection target and the historical collection target correspond to the same cigarette product type; according to the reference distribution data, attention enhancement is carried out on the image feature data to obtain first enhancement data; and determining a defect detection result of the target cigarette product according to the first enhanced data. Through the method, the detection precision of defect detection is improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and particularly to a defect detection method, device, equipment, medium and program product. Background Art

[0002] With the progress of technology and the increasing growth of consumer demands, cigarettes, as fast-moving consumer goods produced on a large scale, have increasingly strict quality control requirements. During the production process of cigarettes, they often need to go through multiple processes. It is precisely due to the technological complexity of cigarettes that diverse types of defects are likely to occur. Therefore, quality control and defect detection during the cigarette production process have become a crucial link.

[0003] Traditional defect detection methods usually involve manual visual inspection of cigarette products. However, the manual visual inspection method is restricted by factors such as the work experience and fatigue level of technicians, and it is difficult to ensure the consistency and stability of detection. To overcome the limitations of the traditional manual visual inspection method, image processing technology has begun to be used for defect detection of cigarette images. However, there is still a problem of low detection accuracy in the traditional defect detection method, which affects the stability of cigarette quality control. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a defect detection method, device, equipment, medium and program product to improve the detection accuracy.

[0005] In a first aspect, the present application provides a defect detection method, including:

[0006] Obtain a captured image of a target cigarette product and extract the image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and,

[0007] Obtain the reference distribution data corresponding to the captured target; the reference distribution data includes the position distribution data of at least one historical captured target in the corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product;

[0008] According to the reference distribution data, perform attention enhancement on the image feature data to obtain first enhanced data;

[0009] According to the first enhanced data, determine the defect detection result of the target cigarette product.

[0010] In one embodiment, according to the reference distribution data, attention enhancement is performed on the image feature data to obtain first enhanced data, including: determining the reference distribution density of historical acquisition targets in different preset image regions according to the reference distribution data; for each preset image region, determining the attention weight of the preset image region according to the reference distribution density corresponding to the preset image region; and performing attention enhancement on the image feature data according to the attention weights corresponding to different preset image regions to obtain first enhanced data.

[0011] In one embodiment, determining the reference distribution density of historical acquisition targets in different preset image regions according to the reference distribution data includes: for each historical acquisition target, determining the center point position of the historical acquisition target according to the position distribution data of the historical acquisition target; determining the center point distribution density of historical acquisition targets in different preset image regions according to the center point positions of the historical acquisition targets; and taking the center point distribution density as the reference distribution density.

[0012] In one embodiment, determining the defect detection result of the target cigarette product according to the first enhanced data includes: obtaining the defect distribution data of the defective cigarette product, where the defect distribution data includes the positions of historical defects in the corresponding historical acquisition images; the defective cigarette product and the target cigarette product correspond to the same cigarette product type; performing attention enhancement on the first enhanced data according to the defect distribution data to obtain second enhanced data; and determining the defect detection result of the target cigarette product according to the second enhanced data.

[0013] In one embodiment, determining the defect detection result of the target cigarette product according to the second enhanced data includes: inputting the second enhanced data into a defect detection model to be trained, and outputting the defect detection result of the target cigarette product; the defect detection result includes a defective category and a non-defective category; using the acquisition image corresponding to the defective category as a training sample of the defect detection model for secondary training of the defect detection model; and / or, in the case where the defect detection result is incorrect, using the corresponding acquisition image as a training sample of the defect detection model for secondary training of the defect detection model.

[0014] In one embodiment, the defect detection result includes a defective category and a non-defective category; the defective category includes the defect type and the defect degree; the defect detection method further includes: in the case where the defect detection result is the defective category, displaying the station to which the target cigarette product belongs and the corresponding defect type, and outputting a warning message corresponding to the defect degree.

[0015] In a second aspect, the present application further provides a defect detection device, including:

[0016] A first acquisition module, configured to acquire a captured image of a target cigarette product and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product;

[0017] A second acquisition module, configured to acquire reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product;

[0018] A processing module, configured to perform attention enhancement on the image feature data according to the reference distribution data to obtain first enhanced data;

[0019] A determination module, configured to determine a defect detection result of the target cigarette product according to the first enhanced data.

[0020] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0021] Acquire a captured image of a target cigarette product and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and,

[0022] Acquire reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product;

[0023] Perform attention enhancement on the image feature data according to the reference distribution data to obtain first enhanced data;

[0024] Determine a defect detection result of the target cigarette product according to the first enhanced data.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Acquire a captured image of a target cigarette product and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and,

[0027] Acquire reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product;

[0028] Enhance the attention of the image feature data according to the reference distribution data to obtain the first enhanced data;

[0029] Determine the defect detection result of the target cigarette product according to the first enhanced data.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0031] Obtain the collected image of the target cigarette product, and extract the image feature data of the collected image; the collected image includes a collection background and a collection target, and the collection target includes the target cigarette product; and,

[0032] Obtain the reference distribution data corresponding to the collection target; the reference distribution data includes the position distribution data of at least one historical collection target in the corresponding historical collection image; the collection target and the historical collection target correspond to the same type of cigarette product;

[0033] Enhance the attention of the image feature data according to the reference distribution data to obtain the first enhanced data;

[0034] Determine the defect detection result of the target cigarette product according to the first enhanced data.

[0035] The above-mentioned defect detection method, device, computer device, computer-readable storage medium and computer program product provide a data basis for subsequent defect detection by obtaining the collected image of the target cigarette product and extracting the image feature data of the collected image. By introducing the reference distribution data, and the reference distribution data can represent the benchmark position distribution of cigarette products under normal production conditions, it is convenient to quickly lock the main area to be detected during the defect detection process, reducing the positioning deviation caused by target offset or background interference. By enhancing the attention of the image feature data according to the reference distribution data, the response of the region strongly related to the historical collection target in the image feature data can be strengthened, focusing on the image region where the main body of the cigarette product is located, and suppressing the interference of the collection background. By determining the defect detection result of the target cigarette product according to the first enhanced data, the detection accuracy is improved, which is beneficial to reducing the false alarm rate, and is also beneficial to enhancing the detection adaptability under different production batches and equipment states. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for describing the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained without creative efforts based on these drawings.

[0037] Figure 1A It is a schematic flow chart of a defect detection method in an embodiment;

[0038] Figure 1B It is a schematic diagram of defect statistics for some workstations in an embodiment;

[0039] Figure 1C It is a structural block diagram of a classification model in an embodiment;

[0040] Figure 2A It is a schematic flow chart of a defect detection method in another embodiment;

[0041] Figure 2B It is a schematic structural diagram of a defect detection system in an embodiment;

[0042] Figure 3 It is a schematic flow chart of the steps for determining a defect detection result in an embodiment;

[0043] Figure 4 It is a schematic flow chart of a defect detection method in yet another embodiment;

[0044] Figure 5 It is a structural block diagram of a defect detection device in an embodiment;

[0045] Figure 6 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, as Figure 1A shown, a defect detection method is provided. In this embodiment, it is exemplified that this method is applied to a terminal. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0048] S110. Obtain a captured image of a target cigarette product, and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product.

[0049] Among them, the target cigarette product may include the output products of different target processes in the cigarette production process or the input materials entering different target processes. The target process may include at least one of a raw material processing process, a cigarette making process, a tipping and making process, and a packaging process. Exemplarily, the packaging process may include at least one of a label pasting process and a packaging printing process. It can be understood that the target cigarette product may include different forms in the cigarette production process, including both the finished products that have completed all processes and the semi-finished products that have not completed the final packaging and the phased products in the middle of the production process. The present application does not make any limitation on the specific product type of the target cigarette product.

[0050] Among them, the acquired image can be understood as the visual image of the target cigarette product. The acquisition target can be understood as the main body of the cigarette product to be defect-detected in the acquired image, that is, the target cigarette product itself. The acquisition background can be understood as the combination of environmental elements in the acquired image other than the acquisition target. The image feature data can be understood as a set of image feature descriptions extracted from the acquired image.

[0051] In an alternative embodiment, the acquired image of the target cigarette product can be obtained through an acquisition device. The acquisition device may include at least one of a camera and a scanner. Exemplarily, at least one detection point may be deployed on the cigarette production line, and an industrial camera may be deployed at each detection point to acquire the acquired image of the target cigarette product.

[0052] S120. Obtain the reference distribution data corresponding to the acquisition target; the reference distribution data includes the position distribution data of at least one historical acquisition target in the corresponding historical acquired image; the acquisition target and the historical acquisition target correspond to the same cigarette product type.

[0053] Among them, the reference distribution data can be understood as the position distribution data of at least one historical acquisition target in the corresponding historical acquired image. It can be understood that during different image acquisition processes, affected by factors such as the position, attitude, and accuracy of the acquisition device, the position of the acquisition target in the image often has differences. By introducing the reference distribution data, the reference position distribution of the cigarette product can be determined, so that the main body area to be detected can be quickly locked in the subsequent defect detection process, reducing the positioning deviation caused by target offset or background interference.

[0054] S130. According to the reference distribution data, perform attention enhancement on the image feature data to obtain the first enhanced data.

[0055] In an alternative embodiment, the reference distribution density of historical acquisition targets in different preset image regions may be determined according to reference distribution data; for each preset image region, the attention weight of the preset image region may be determined according to the reference distribution density corresponding to the preset image region; and the image feature data may be enhanced by attention according to the attention weights corresponding to different preset image regions to obtain first enhanced data.

[0056] Among them, the preset image region a n can be understood as the nth image region pre-divided in the historical acquisition image, where n represents the index of the preset image region. The image sizes of the historical acquisition image and the acquisition image of the target cigarette product may be the same.

[0057] Among them, the preset image region a n The corresponding reference distribution density can be understood as the density of the central points of different historical acquisition targets distributed in the preset image region a n inside.

[0058] In an alternative implementation manner, for each historical acquisition target, the central point position of the historical acquisition target may be determined according to the position distribution data of the historical acquisition target; according to the central point positions of each historical acquisition target, the central point distribution density of the historical acquisition targets in different preset image regions may be determined; and the central point distribution density may be used as the reference distribution density.

[0059] Optionally, the position distribution data may include the position coordinates of each sampling point in the historical acquisition target.

[0060] Optionally, the corresponding historical acquisition target may be marked in the historical acquisition image in the form of a marked box. Correspondingly, the position distribution data may include the position coordinates of the marked box where the historical acquisition target is located.

[0061] Exemplarily, the mean value of the position distribution data of the historical acquisition target may be taken to obtain the central point position of the historical acquisition target; the central point positions of each historical acquisition target may be normalized to obtain the central point position distributions of each; and according to the central point position distributions of each, the central point distribution density of the historical acquisition targets in different preset image regions may be determined.

[0062] Exemplarily, the KED (Kernel Density Estimation) method may be used to establish a 2D (two-dimensional) probability distribution model. Correspondingly, the central point positions of each historical acquisition target may be input into the 2D (two-dimensional) probability distribution model to obtain the central point distribution density of the historical acquisition targets in different preset image regions.

[0063] In an alternative embodiment, the reference distribution densities corresponding to each preset image region may be summed to obtain the total distribution density; the proportion of the reference distribution density corresponding to the preset image region in the total distribution density is used as the attention weight of the preset image region.

[0064] In another alternative embodiment, a spatial probability weight map may be generated according to the distribution density of the center points of the historical acquisition targets in different preset image regions; according to the spatial probability weight map, the attention weights of each preset image region are determined. Among them, the spatial probability weight map, that is, the probability heat map, is used to represent the attention weights of different preset image regions.

[0065] In an alternative implementation manner, the image feature data may be divided into multiple image feature sub-data according to the preset image regions, and each image feature sub-data corresponds to a preset image region; for each preset image region, the image feature sub-data corresponding to the preset image region and the corresponding attention weight are subjected to convolution processing to obtain a first enhanced sub-data; the first enhanced sub-data corresponding to each preset image region are aggregated to obtain the first enhanced data.

[0066] S140. Determine the defect detection result of the target cigarette product according to the first enhanced data.

[0067] In an alternative embodiment, the first enhanced data may be input into a defect detection model to determine the defect detection result of the target cigarette product. Among them, the defect detection model may be a traditional machine learning model or a neural network model. The present application does not make any limitation on the specific model type of the defect detection model.

[0068] Among them, the defect detection result may include a defective category and a non-defective category; the defective category may include the defect type and the defect degree. The defect type may be understood as the category to which the defect of the target cigarette product belongs, and the defect degree is used to characterize the severity of the defect of the target cigarette product. Exemplarily, the defect type may include at least one of label damage, label missing, and packaging damage, etc. Exemplarily, the defect degree may include minor defects, moderate defects, and severe defects.

[0069] In an alternative embodiment, the defect degree may be determined according to the defect area of the target cigarette product. Exemplarily, when the defect area is less than the first area threshold, it is determined that the defect degree is a minor defect; when the defect area is greater than the second area threshold, it is determined that the defect degree is a severe defect; when the defect area is not less than the first area threshold and not greater than the second area threshold, it is determined that the defect degree is a moderate defect. The present application does not make any limitation on the specific determination method of the defect degree.

[0070] In an optional embodiment, a warning message may be output when the defect frequency exceeds a preset frequency threshold or when the defect level is a severe defect. It should be noted that the preset frequency threshold can be set by technicians according to needs or experience, or determined through a large number of experiments, and the present application does not make any limitations thereto.

[0071] In another optional embodiment, a warning score may be determined based on at least one of the defect type, defect level, and defect frequency of the target cigarette product; a warning message is output when the warning score is greater than a preset score threshold. Among them, the warning message may include at least one of the defect type, defect level, defect frequency, warning score, etc. Optionally, corresponding reference scores may be predefined for different defect types, defect levels, and defect frequencies respectively to obtain a reference score comparison table; based on the reference score comparison table, the reference scores of the target cigarette product under the corresponding scoring types are queried, and the reference scores under different scoring types are summed to obtain the warning score.

[0072] Exemplarily, the warning message may further include a warning level, and the warning level is determined by the warning score. It should be noted that the preset score threshold can be set by technicians according to needs or experience, or determined through a large number of experiments, and the present application does not make any limitations thereto.

[0073] In an optional embodiment, when the defect detection result is a defective category, the station to which the target cigarette product belongs and the corresponding defect type may be displayed, and a warning message corresponding to the defect level is output. Exemplarily, when the defect detection result is a defective category, a defect distribution statistical chart of different stations may also be displayed to facilitate subsequent defect analysis and tracking.

[0074] Optionally, the above-mentioned defect type, warning message, and defect distribution statistical chart may be displayed on the defect detection interface. In addition, the defect detection interface may also display at least one of real-time defect statistics and warning pop-ups. At the same time, historical defect data can also be viewed through the defect detection interface. It should be noted that the present application does not make any limitations on the specific interface content and specific presentation method of the defect detection interface. Exemplarily, the real-time defect statistics may be presented by a bar chart or a column chart, etc. Refer to Figure 1B is a schematic diagram of defect statistics for some stations in an embodiment. Among them, Figure 1B exemplarily shows the defect statistics of two stations, where the abscissa represents time and the ordinate represents the number of defects. By displaying the defect statistical data on the defect detection interface, it is convenient for technicians to intuitively obtain the overall defect situation.

[0075] In an optional embodiment, the above defect detection method can be implemented by a classification model. Exemplarily, the classification model can be a Yolov8 model with a spatial attention module introduced. Refer to Figure 1C As shown in the structural block diagram of the classification model, the classification model includes a backbone network and a detection head network Head. The backbone network is used to extract the image feature data of the acquired image. The detection head network includes an adaptive average pooling layer, a probability-guided spatial attention module (PSSA, Progressive Global Sampling Attention), and a classification module. Among them, the adaptive average pooling layer is used to process the reference distribution data and the image feature data; the spatial attention module is used to enhance the attention of the image feature data according to the reference distribution data to obtain first enhanced data; the classification module is used to determine the defect detection result of the target cigarette product according to the first enhanced data.

[0076] The above defect detection method provides a data basis for subsequent defect detection by acquiring the acquired image of the target cigarette product and extracting the image feature data of the acquired image. By introducing reference distribution data, and the reference distribution data can represent the benchmark position distribution of cigarette products under normal production conditions, it is convenient to quickly lock the main area to be detected during the defect detection process and reduce the positioning deviation caused by target offset or background interference. By enhancing the attention of the image feature data according to the reference distribution data, the response of the region strongly related to the historical acquisition target in the image feature data can be strengthened, focusing on the image region where the main body of the cigarette product is located, and suppressing the interference of the acquisition background. By determining the defect detection result of the target cigarette product according to the first enhanced data, the detection accuracy is improved, which is beneficial to reducing the false alarm rate and also beneficial to enhancing the detection adaptability under different production batches and equipment states.

[0077] In an optional embodiment, the above defect detection method can also be implemented through the interaction between the server and the terminal. Among them, the terminal can include a first terminal and a second terminal. As Figure 2A shown, it includes the following steps:

[0078] S210. The first terminal acquires the acquired image of the target cigarette product.

[0079] Among them, the acquired image of the target cigarette product can be acquired by an acquisition device. The acquired image includes an acquisition background and an acquisition target, and the acquisition target includes the target cigarette product.

[0080] S220. The first terminal sends the acquired image to the server.

[0081] S230. The server extracts the image feature data of the acquired image.

[0082] S240. The server obtains the reference distribution data corresponding to the acquisition target; the reference distribution data includes the position distribution data of at least one historical acquisition target in the corresponding historical acquisition image; the acquisition target and the historical acquisition target correspond to the same type of cigarette product.

[0083] S250. The server enhances the attention of the image feature data according to the reference distribution data to obtain the first enhanced data.

[0084] S260. The server determines the defect detection result of the target cigarette product according to the first enhanced data.

[0085] S270. The second terminal sends a result query request to the server.

[0086] S280. The server sends the defect detection result to the second terminal in response to the result query request of the second terminal.

[0087] In an optional embodiment, the first terminal may be connected to at least one acquisition device for obtaining the acquisition image of the target cigarette product acquired by the acquisition device. The first terminal may upload the acquisition image to the server through a wireless network. The second terminal may access the server through a wireless network to obtain the defect detection result. Exemplarily, the first terminal may include a vision computer, and the second terminal may include at least one of a mobile terminal and a PC (Personal Computer). The present application does not make any limitation on the specific types of the first terminal and the second terminal. Optionally, the first terminal and the second terminal may share a terminal device.

[0088] In an optional embodiment, the second terminal may log in to the server and display a defect detection interface when the authentication is passed. Among them, at least one of the workstations to which the foregoing target cigarette product belongs, the corresponding defect types, warning information, and defect distribution statistical charts may be displayed on the defect detection interface.

[0089] It should be noted that at least one acquisition device may be deployed at different workstations. Correspondingly, multiple first terminals may also be provided and respectively deployed at different workstations, so as to realize defect detection of target cigarette products at different workstations under a distributed architecture. Under the distributed architecture, it is convenient to perform unified and centralized defect detection on cigarette products corresponding to different workstations, convenient to obtain the global defect distribution, and also provides convenience for subsequent defect analysis and defect tracking.

[0090] Reference Figure 2BThe figure shows a schematic structural diagram of a defect detection system. The defect detection system includes N vision computers, and each vision computer corresponds to at least one acquisition device. As a first terminal, the vision computer can obtain the acquisition images of the target cigarette products collected by the corresponding acquisition device, and upload them to the server through the WIFI (Wireless Fidelity) function of the workshop router; the above classification model can be configured in the server to perform defect detection on the uploaded acquisition images to determine the defect detection results of the target cigarette products. The PC or mobile terminal, as a second terminal, can access the network application through the WIFI function and receive the defect detection results sent by the server. Of course, the PC or mobile terminal can also display the defect detection interface through the network application. The defect display interface can display at least one of the defect types, warning information, defect distribution statistical charts, real-time defect statistics, warning pop-ups, and historical defect data, etc.

[0091] Based on the technical solutions of the above embodiments, the present application also provides an alternative embodiment, in which the steps for determining the defect detection results are refined.

[0092] See Figure 3 The steps for determining the defect detection results shown in the figure include:

[0093] S310. Obtain the defect distribution data of the defective cigarette products. The defect distribution data includes the positions of historical defects in the corresponding historical acquisition images; the defective cigarette products and the target cigarette products correspond to the same cigarette product type.

[0094] S320. Perform attention enhancement on the first enhanced data according to the defect distribution data to obtain the second enhanced data.

[0095] In an alternative embodiment, the defect distribution density of historical defects in different preset image regions can be determined according to the defect distribution data; for each preset image region, the defect weight of the preset image region can be determined according to the defect distribution density corresponding to the preset image region; and the first enhanced data can be subjected to attention enhancement according to the defect weights corresponding to different preset image regions to obtain the second enhanced data.

[0096] Optionally, for each historical defect, the central point position of the historical defect in the historical acquisition image can be determined; according to the central point positions of the historical defects, the central point distribution density of historical defects in different preset image regions can be determined; and the central point distribution density of the historical defects can be used as the defect distribution density.

[0097] Optionally, the defect distribution densities corresponding to the preset image regions can be summed to obtain the total defect density; and the proportion of the defect distribution density corresponding to the preset image region in the total defect density can be used as the defect weight of the preset image region.

[0098] As can be seen from the foregoing, the first enhanced data includes first enhanced sub-data corresponding to different preset image regions; correspondingly, the first enhanced sub-data corresponding to the preset image regions can be convolved with the corresponding defect weights to obtain second enhanced sub-data; the second enhanced sub-data corresponding to each preset image region is aggregated to obtain second enhanced data.

[0099] S330. Determine the defect detection result of the target cigarette product according to the second enhanced data.

[0100] In an alternative embodiment, the second enhanced data can be input into a defect detection model to be trained, and the defect detection result of the target cigarette product is output; the defect detection result includes a defective category and a non-defective category.

[0101] In an alternative embodiment, the second enhanced data can be input into the defect detection model to determine the defect detection result of the target cigarette product. Among them, the defect detection model can be a traditional machine learning model or a neural network model. The present application does not make any limitation on the specific model type of the defect detection model.

[0102] In an alternative embodiment, the collected image corresponding to the defective category can be used as a training sample of the defect detection model for secondary training of the defect detection model; and / or, in the case where the defect detection result is incorrect, the corresponding collected image can be used as a training sample of the defect detection model for secondary training of the defect detection model.

[0103] Optionally, the training process of the defect detection model can include a pre-training stage, a training stage, and an optimization stage.

[0104] For the pre-training stage, a publicly available dataset can be used to pre-train the defect detection model. Exemplarily, the publicly available dataset can include ImageNet (Image Network). Among them, ImageNet is a visual recognition benchmark dataset, including annotated images and hierarchical semantic structures.

[0105] For the training stage, a dedicated dataset can be used to train the pre-trained defect detection model. Exemplarily, the dedicated dataset can include normal sample images and various defect sample images of cigarette products.

[0106] For the optimization stage, the collected images corresponding to the defective types identified by the defect detection model and the collected images corresponding to the incorrect defect detection results can be used as training samples to retrain the trained defect detection model to improve the model accuracy. Exemplarily, the accuracy of the model can be tested, and when the accuracy of the model is greater than the preset accuracy threshold, the training of the defect detection model is completed; otherwise, the defect detection model is retrained.

[0107] In the above steps, by introducing the defect distribution data and enhancing the attention of the first enhanced data according to the defect distribution data, the regional response related to the historical defect position in the first enhanced data can be strengthened. By determining the defect detection result of the target cigarette product according to the second enhanced data, the detection accuracy is improved, which is beneficial to reducing the false alarm rate.

[0108] Based on the technical solutions of the above embodiments, the present application also provides another optional embodiment, in which the defect detection method is described in detail.

[0109] See Figure 4 The defect detection method shown includes:

[0110] S410. Obtain the collected image of the target cigarette product and extract the image feature data of the collected image; wherein, the collected image includes a collection background and a collection target, and the collection target includes the target cigarette product.

[0111] S420. Obtain the reference distribution data corresponding to the collection target; the reference distribution data includes the position distribution data of at least one historical collection target in the corresponding historical collection image; the collection target and the historical collection target correspond to the same type of cigarette product.

[0112] S430. For each historical collection target, determine the center point position of the historical collection target according to the position distribution data of the historical collection target.

[0113] S440. According to the center point positions of the historical collection targets, determine the center point distribution density of the historical collection targets in different preset image regions; use the center point distribution density as the reference distribution density.

[0114] S450. For each preset image region, determine the attention weight of the preset image region according to the reference distribution density corresponding to the preset image region.

[0115] S460. According to the attention weights corresponding to different preset image regions, perform attention enhancement on the image feature data to obtain the first enhanced data.

[0116] S470. Obtain the defect distribution data of defective cigarette products, where the defect distribution data includes the positions of historical defects in the corresponding historical acquisition images; the defective cigarette products and the target cigarette products correspond to the same type of cigarette products.

[0117] S480. According to the defect distribution data, perform attention enhancement on the first enhanced data to obtain the second enhanced data.

[0118] S490. Determine the defect detection result of the target cigarette products according to the second enhanced data.

[0119] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0120] Based on the same inventive concept, the embodiments of the present application also provide a defect detection device for implementing the above-mentioned defect detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more of the following defect detection device embodiments can refer to the limitations on the defect detection method in the above text, and will not be repeated here.

[0121] In an exemplary embodiment, as Figure 5 shown, a defect detection device is provided, including: a first acquisition module 510, a second acquisition module 520, a processing module 530, and a determination module 540, where:

[0122] The first acquisition module 510 is configured to acquire the acquisition image of the target cigarette products and extract the image feature data of the acquisition image; the acquisition image includes an acquisition background and an acquisition target, and the acquisition target includes the target cigarette products.

[0123] The second acquisition module 520 is configured to acquire the reference distribution data corresponding to the acquisition target; the reference distribution data includes the position distribution data of at least one historical acquisition target in the corresponding historical acquisition image; the acquisition target and the historical acquisition target correspond to the same type of cigarette products.

[0124] A processing module 530, configured to perform attention enhancement on the image feature data according to the reference distribution data to obtain first enhanced data.

[0125] A determination module 540, configured to determine a defect detection result of the target cigarette product according to the first enhanced data.

[0126] In one embodiment, the processing module includes: a first determination unit, configured to determine the reference distribution density of the historical acquisition targets in different preset image regions according to the reference distribution data; a second determination unit, configured to determine the attention weight of each preset image region according to the reference distribution density corresponding to the preset image region; a first enhancement unit, configured to perform attention enhancement on the image feature data according to the attention weights corresponding to different preset image regions to obtain first enhanced data.

[0127] In one embodiment, the first determination unit includes: a first determination subunit, configured to determine the center point position of each historical acquisition target according to the position distribution data of the historical acquisition target; a second determination subunit, configured to determine the center point distribution density of the historical acquisition targets in different preset image regions according to the center point positions of the historical acquisition targets; and use the center point distribution density as the reference distribution density.

[0128] In one embodiment, the determination module includes: a first acquisition unit, configured to acquire defect distribution data of a defective cigarette product, where the defect distribution data includes the positions of historical defects in corresponding historical acquisition images; the defective cigarette product and the target cigarette product correspond to the same cigarette product type; a second enhancement unit, configured to perform attention enhancement on the first enhanced data according to the defect distribution data to obtain second enhanced data; a third determination unit, configured to determine a defect detection result of the target cigarette product according to the second enhanced data.

[0129] In one embodiment, the third determination unit includes: an input subunit, configured to input the second enhanced data into a to-be-trained defect detection model and output a defect detection result of the target cigarette product; the defect detection result includes a defective category and a non-defective category; a first training subunit, configured to use the acquisition image corresponding to the defective category as a training sample of the defect detection model to perform secondary training on the defect detection model; and / or, a second training subunit, configured to use the corresponding acquisition image as a training sample of the defect detection model to perform secondary training on the defect detection model when the defect detection result is incorrect.

[0130] In one embodiment, the defect detection result includes a defective category and a non-defective category; the defective category includes the defect type and the defect degree; the defect detection device further includes a display module, which is configured to display the station to which the target cigarette product belongs and the corresponding defect type when the defect detection result is the defective category, and output a warning message corresponding to the defect degree.

[0131] Each module in the above-mentioned defect detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in the form of hardware or be independent of it, or be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0132] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for the processor to exchange information with external devices. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements a defect detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0133] Those skilled in the art can understand that Figure 6 the structure shown in

[0134] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0135] Obtain a captured image of a target cigarette product and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and,

[0136] Obtain reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same cigarette product type;

[0137] According to the reference distribution data, perform attention enhancement on the image feature data to obtain first enhanced data;

[0138] According to the first enhanced data, determine a defect detection result of the target cigarette product.

[0139] In an embodiment, when the processor executes the computer program, the following steps are further implemented: according to the reference distribution data, determine the reference distribution density of historical captured targets in different preset image regions; for each preset image region, determine the attention weight of the preset image region according to the reference distribution density corresponding to the preset image region; according to the attention weights corresponding to different preset image regions, perform attention enhancement on the image feature data to obtain first enhanced data.

[0140] In an embodiment, when the processor executes the computer program, the following steps are further implemented: for each historical captured target, determine the center point position of the historical captured target according to the position distribution data of the historical captured target; according to the center point positions of the historical captured targets, determine the center point distribution density of historical captured targets in different preset image regions; use the center point distribution density as the reference distribution density.

[0141] In an embodiment, when the processor executes the computer program, the following steps are further implemented: obtain defect distribution data of a defective cigarette product, where the defect distribution data includes the positions of historical defects in corresponding historical captured images; the defective cigarette product and the target cigarette product correspond to the same cigarette product type; according to the defect distribution data, perform attention enhancement on the first enhanced data to obtain second enhanced data; according to the second enhanced data, determine a defect detection result of the target cigarette product.

[0142] In one embodiment, when the processor executes the computer program, the following steps are further implemented: input the second enhanced data into the defect detection model to be trained, and output the defect detection result of the target cigarette product; the defect detection result includes a defective category and a non-defective category; use the captured image corresponding to the defective category as a training sample of the defect detection model for secondary training of the defect detection model; and / or, in the case where the defect detection result is incorrect, use the corresponding captured image as a training sample of the defect detection model for secondary training of the defect detection model.

[0143] In one embodiment, the defect detection result includes a defective category and a non-defective category; the defective category includes the defect type and the defect degree; when the processor executes the computer program, the following steps are further implemented: in the case where the defect detection result is a defective category, display the station to which the target cigarette product belongs and the corresponding defect type, and output a warning message corresponding to the defect degree.

[0144] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0145] Obtain the captured image of the target cigarette product, and extract the image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and,

[0146] Obtain the reference distribution data corresponding to the captured target; the reference distribution data includes the position distribution data of at least one historical captured target in the corresponding historical captured image; the captured target and the historical captured target correspond to the same cigarette product type;

[0147] According to the reference distribution data, perform attention enhancement on the image feature data to obtain the first enhanced data;

[0148] According to the first enhanced data, determine the defect detection result of the target cigarette product.

[0149] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: according to the reference distribution data, determine the reference distribution density of the historical captured target in different preset image regions; for each preset image region, determine the attention weight of the preset image region according to the reference distribution density corresponding to the preset image region; according to the attention weights corresponding to different preset image regions, perform attention enhancement on the image feature data to obtain the first enhanced data.

[0150] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for each historical acquisition target, determine the center point position of the historical acquisition target according to the position distribution data of the historical acquisition target; according to the center point positions of the historical acquisition targets, determine the center point distribution density of the historical acquisition targets in different preset image regions; and use the center point distribution density as the reference distribution density.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtain the defect distribution data of defective cigarette products, where the defect distribution data includes the positions of historical defects in the corresponding historical acquisition images; the defective cigarette products and the target cigarette products correspond to the same type of cigarette products; according to the defect distribution data, perform attention enhancement on the first enhanced data to obtain second enhanced data; and determine the defect detection result of the target cigarette products according to the second enhanced data.

[0152] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: input the second enhanced data into a defect detection model to be trained, and output the defect detection result of the target cigarette products; the defect detection result includes a defective category and a non-defective category; use the acquisition image corresponding to the defective category as a training sample for the defect detection model to perform secondary training on the defect detection model; and / or, in the case where the defect detection result is incorrect, use the corresponding acquisition image as a training sample for the defect detection model to perform secondary training on the defect detection model.

[0153] In one embodiment, the defect detection result includes a defective category and a non-defective category; the defective category includes the defect type and the defect degree; when the computer program is executed by a processor, the following steps are further implemented: in the case where the defect detection result is a defective category, display the station to which the target cigarette product belongs and the corresponding defect type, and output a warning message corresponding to the defect degree.

[0154] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor, implements the following steps:

[0155] Obtain the acquisition image of the target cigarette product, and extract the image feature data of the acquisition image; the acquisition image includes an acquisition background and an acquisition target, and the acquisition target includes the target cigarette product; and,

[0156] Obtain the reference distribution data corresponding to the acquisition target; the reference distribution data includes the position distribution data of at least one historical acquisition target in the corresponding historical acquisition image; the acquisition target and the historical acquisition target correspond to the same type of cigarette products;

[0157] According to the reference distribution data, perform attention enhancement on the image feature data to obtain the first enhanced data;

[0158] Determine the defect detection result of the target cigarette product according to the first enhanced data.

[0159] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determine the reference distribution density of historical acquisition targets in different preset image regions according to the reference distribution data; for each preset image region, determine the attention weight of the preset image region according to the reference distribution density corresponding to the preset image region; perform attention enhancement on the image feature data according to the attention weights corresponding to different preset image regions to obtain the first enhanced data.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for each historical acquisition target, determine the center point position of the historical acquisition target according to the position distribution data of the historical acquisition target; determine the center point distribution density of historical acquisition targets in different preset image regions according to the center point positions of each historical acquisition target; use the center point distribution density as the reference distribution density.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtain the defect distribution data of the defective cigarette product, where the defect distribution data includes the positions of historical defects in the corresponding historical acquisition images; the defective cigarette product and the target cigarette product correspond to the same cigarette product type; perform attention enhancement on the first enhanced data according to the defect distribution data to obtain the second enhanced data; determine the defect detection result of the target cigarette product according to the second enhanced data.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: input the second enhanced data into the defect detection model to be trained, and output the defect detection result of the target cigarette product; the defect detection result includes a defective category and a non-defective category; use the acquisition image corresponding to the defective category as the training sample of the defect detection model for secondary training of the defect detection model; and / or, in the case where the defect detection result is incorrect, use the corresponding acquisition image as the training sample of the defect detection model for secondary training of the defect detection model.

[0163] In one embodiment, the defect detection result includes a defective category and a non-defective category; the defective category includes the defect type and the defect degree; when the computer program is executed by a processor, the following steps are further implemented: in the case where the defect detection result is the defective category, display the station to which the target cigarette product belongs and the corresponding defect type, and output a warning message corresponding to the defect degree.

[0164] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0165] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in this application.

[0166] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A defect detection method, characterized in that, The method includes: Obtaining a captured image of a target cigarette product and extracting image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and, Obtaining reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product; Enhancing the attention of the image feature data according to the reference distribution data to obtain first enhanced data; Determining a defect detection result of the target cigarette product according to the first enhanced data.

2. The method according to claim 1, wherein The enhancing the attention of the image feature data according to the reference distribution data to obtain first enhanced data includes: Determining a reference distribution density of historical captured targets in different preset image regions according to the reference distribution data; For each preset image region, determining an attention weight of the preset image region according to the reference distribution density corresponding to the preset image region; Enhancing the attention of the image feature data according to the attention weights corresponding to different preset image regions to obtain first enhanced data.

3. The method according to claim 2, characterized in that, The determining a reference distribution density of historical captured targets in different preset image regions according to the reference distribution data includes: For each historical captured target, determining a center point position of the historical captured target according to the position distribution data of the historical captured target; Determining a center point distribution density of historical captured targets in different preset image regions according to the center point positions of the historical captured targets; Taking the center point distribution density as the reference distribution density.

4. The method according to any one of claims 1-3, characterized in that The determining a defect detection result of the target cigarette product according to the first enhanced data includes: Obtaining defect distribution data of defective cigarette products, where the defect distribution data includes the positions of historical defects in corresponding historical captured images; the defective cigarette products and the target cigarette products correspond to the same type of cigarette product; Enhancing the attention of the first enhanced data according to the defect distribution data to obtain second enhanced data; Determining a defect detection result of the target cigarette product according to the second enhanced data.

5. The method according to claim 4, wherein The determining a defect detection result of the target cigarette product according to the second enhanced data includes: Inputting the second enhanced data into a defect detection model to be trained and outputting a defect detection result of the target cigarette product; the defect detection result includes a defective category and a non-defective category; Taking the captured image corresponding to the defective category as a training sample of the defect detection model for secondary training of the defect detection model; and / or, In the case where the defect detection result is incorrect, taking the corresponding captured image as a training sample of the defect detection model for secondary training of the defect detection model.

6. The method according to any one of claims 1 to 3, characterized in that The defect detection result includes a defective category and a non-defective category; the defective category includes a defect type and a defect degree; the method further includes: In the case where the defect detection result is a defective category, display the station to which the target cigarette product belongs and the corresponding defect types, and output a warning message corresponding to the defect degree.

7. A defect detection device, characterized in that, The device includes: A first acquisition module, configured to acquire a captured image of a target cigarette product and extract image feature data of the captured image; the captured image includes a captured background and a captured target, and the captured target includes the target cigarette product; and, A second acquisition module, configured to acquire reference distribution data corresponding to the captured target; the reference distribution data includes position distribution data of at least one historical captured target in a corresponding historical captured image; the captured target and the historical captured target correspond to the same type of cigarette product; A processing module, configured to perform attention enhancement on the image feature data according to the reference distribution data to obtain first enhanced data; A determination module, configured to determine a defect detection result of the target cigarette product according to the first enhanced data.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images