Tobacco product detection method and device, computer equipment and storage medium
By combining multimodal data fusion processing of image features and point cloud features, the problems of inaccurate detection results and high cost in existing tobacco product detection methods are solved, and high-accuracy and robust detection in complex scenarios is achieved.
Patent Information
- Application Number
- CN202510831365.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-20
- Publication Date
- 2025-09-23
AI Technical Summary
Existing tobacco product detection methods rely on camera quality, resulting in inaccurate and costly detection results and insufficient robustness in complex scenarios.
By acquiring the target image features and target point cloud features of tobacco products, convolutional neural networks and point cloud extraction models are used to perform feature extraction and fusion processing, and multimodal data is combined for detection.
The accuracy and robustness of tobacco product detection are improved, especially maintaining high accuracy under low light or occlusion conditions.
Smart Images
Figure CN120689710A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automation technology, and in particular to a tobacco product detection method, device, computer equipment, and storage medium. Background Art
[0002] With the rapid development of fields such as artificial intelligence and machine vision, product production is gradually shifting to unmanned production methods. In unmanned production methods, automated product inspection is a core link.
[0003] Currently, visual inspection is the most common method used for product inspection. For example, in the tobacco production process, visual inspection can capture product images and perform inspections based on these images. However, this visual inspection method relies heavily on the quality of the camera and algorithm. Poor-quality cameras, due to the poor image quality of the captured images, can lead to inaccurate product detection results. Using higher-quality cameras is often more expensive, significantly increasing costs. Summary of the Invention
[0004] Based on this, it is necessary to provide a tobacco product detection method, device, computer equipment and storage medium that can improve the accuracy of tobacco product detection in response to the above technical problems.
[0005] In a first aspect, the present application provides a method for detecting tobacco products. The method comprises:
[0006] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0007] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0008] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0009] In one embodiment, obtaining target image features and target point cloud features of a tobacco product to be inspected includes:
[0010] Obtaining initial image information and initial point cloud data of the tobacco product to be inspected;
[0011] The initial image information is extracted and processed based on a convolutional neural network to obtain the target image features of the tobacco product to be detected;
[0012] The initial point cloud data is extracted and processed based on the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected.
[0013] In one embodiment, the target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected, including:
[0014] Based on the neural network model, the target image features and target point cloud features are fused to obtain the fused features of the tobacco products to be detected.
[0015] In one embodiment, determining a target detection result of a tobacco product to be detected based on the fusion features includes:
[0016] Input the fused features into the target detection model to obtain the initial detection results of the tobacco products to be detected;
[0017] Determining the presence of a first defect in the initial inspection result;
[0018] A target detection result of the tobacco product to be detected is determined according to the presence of the first defect, the target image feature and the initial detection result.
[0019] In one embodiment, determining a target detection result of the tobacco product to be inspected based on the presence of the first defect, target image features, and the initial detection result includes:
[0020] When the first defect existence condition is that a defect exists, determining the defect type of the tobacco product to be inspected;
[0021] Determine the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results.
[0022] In one embodiment, determining a target detection result of a tobacco product to be inspected based on a defect type, target image features, and an initial detection result includes:
[0023] When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the presence of the second defect is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected;
[0024] When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
[0025] In a second aspect, the present application also provides a tobacco product detection device. The device includes:
[0026] An acquisition module, used to acquire target image features and target point cloud features of the tobacco product to be detected;
[0027] The first determination module is used to fuse the target image features and the target point cloud features to obtain the fused features of the tobacco product to be detected;
[0028] The second determination module is used to determine the target detection result of the tobacco product to be detected based on the fusion feature and the target image feature.
[0029] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are performed:
[0030] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0031] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0032] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:
[0034] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0035] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0036] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0037] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:
[0038] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0039] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0040] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0041] The above-mentioned tobacco product detection method, device, computer equipment and storage medium obtain the target image features and target point cloud features of the tobacco product to be detected. The target image features and target point cloud features are fused to obtain the fused features of the tobacco product to be detected. According to the fused features and the target image features, the target detection results of the tobacco product to be detected are determined. The present application obtains the fused features of the tobacco product to be detected by fusing the target image features and the target point cloud features, and determines the target detection results of the tobacco product to be detected based on the fused features. Through the fusion of multimodal data, not only the accuracy of the target detection results for the tobacco product to be detected is effectively improved, but also the robustness of detecting the tobacco product to be detected in complex scenes is improved, and it has high accuracy even in low light or occlusion conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A diagram illustrating the application environment of the tobacco product detection method provided in this embodiment;
[0043] Figure 2 A schematic flow chart of the first tobacco product detection method provided in this embodiment;
[0044] Figure 3 A schematic diagram of the process of determining target point cloud features of tobacco products to be detected provided in this embodiment;
[0045] Figure 4 A schematic diagram of a process for determining target detection results of a tobacco product to be detected provided in this embodiment;
[0046] Figure 5 A schematic flow chart of a second tobacco product detection method provided in this embodiment;
[0047] Figure 6 A structural block diagram of a tobacco product detection device provided in this embodiment;
[0048] Figure 7 This is a diagram of the internal structure of the computer device provided in this embodiment. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0050] The tobacco product detection method provided in the embodiments of the present application can be applied to Figure 1In the application environment shown, the detection device obtains the target image features and target point cloud features of the tobacco product to be detected through the sensing device. The detection device fuses the target image features and target point cloud features to obtain a fused feature of the tobacco product to be detected. Based on the fused feature and the target image features, the detection device determines the target detection result of the tobacco product to be detected.
[0051] The detection device refers to a device that can be used to detect tobacco products to be detected, for example, it can be a host device or a server. The server can be a single server or a server cluster.
[0052] A sensing device is a device that can acquire image information and point cloud data of the tobacco product being inspected. For example, it can include a camera to acquire image information of the tobacco product being inspected. It can also include a laser radar to acquire point cloud data of the tobacco product being inspected.
[0053] In one embodiment, Figure 2 As shown, a tobacco product detection method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the detection equipment in the example:
[0054] S201, obtaining target image features and target point cloud features of the tobacco product to be detected.
[0055] Among them, the tobacco products to be tested refer to products that have not yet been tested. In this application, the tobacco products to be tested mainly refer to tobacco products on the production line. They can be all products on the production line, or products selected randomly or according to preset rules on the production line. The target image features refer to the relevant image features of the tobacco products to be tested that meet the requirements for detecting the tobacco products to be tested. The target point cloud features refer to the relevant point cloud features of the tobacco products to be tested that meet the requirements for detecting the tobacco products to be tested.
[0056] Optionally, in this embodiment, target image features of the tobacco product to be detected are acquired based on a camera.
[0057] Optionally, in this embodiment, target point cloud features of the tobacco product to be detected are obtained based on a laser radar.
[0058] Optionally, in this embodiment, an information acquisition request is sent to the sensing device, so that the sensing device feeds back target image features and target point cloud features of the tobacco product to be detected to the detection device based on the information acquisition request.
[0059] S202 , fusing target image features and target point cloud features to obtain fused features of the tobacco product to be detected.
[0060] As an optional implementation method of the embodiment of the present application, based on the Early Fusion technology, the target image features and the target point cloud features are spliced at the input stage to obtain the target point cloud features of the tobacco product to be detected. This method can utilize the original information of the two modalities, but due to the high data dimension, it may result in a heavy computational burden.
[0061] Another optional implementation method of the present application is to fuse the target image features and the target point cloud features based on mid-level fusion technology, that is, based on the intermediate layer of the neural network model to obtain the fused features of the tobacco product to be detected. Specifically, the neural network model includes a first weight coefficient for the target image features and a second weight coefficient for the target point cloud features. Based on the first weight coefficient and the second weight coefficient for the target point cloud features, the target image features and the target point cloud features are weightedly fused to obtain the fused features of the tobacco product to be detected. This method combines the information advantages of the two modalities by fusing the feature representations of each modality. The target image features and the target point cloud features are usually combined using weighted averaging or other algorithms.
[0062] Another optional implementation of this application embodiment is to fuse the target image features and target point cloud features based on late fusion to obtain fused features of the tobacco product to be detected. This method can simplify calculations, but may lose some of the performance improvements brought by the fusion process.
[0063] S203: Determine the target detection result of the tobacco product to be detected based on the fusion feature and the target image feature.
[0064] As an optional implementation of the embodiment of the present application, the fusion features are input into the target detection model, and the target detection model outputs the target detection result of the tobacco product to be detected.
[0065] As another optional implementation of the embodiment of the present application, the fusion feature is input into the target detection model, and the target detection model outputs a first detection result. The target image feature is input into the target detection model to obtain a second detection feature. The target detection result is determined based on the first detection result and the second detection result. Exemplarily, if the first detection result and the second detection result both include the position information of the tobacco product to be detected, the intermediate position information between the position information of the tobacco product to be detected contained in the first detection result and the position information of the tobacco product to be detected contained in the second detection result is used as the position information of the tobacco product to be detected in the target detection product.
[0066] In this embodiment, the target image features and target point cloud features of the tobacco product to be detected are obtained. The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected. Based on the fused features and the target image features, the target detection results of the tobacco product to be detected are determined. The present application obtains the fused features of the tobacco product to be detected by fusing the target image features and the target point cloud features, and determines the target detection results of the tobacco product to be detected based on the fused features. Through the fusion of multimodal data, not only the accuracy of the target detection results for the tobacco product to be detected is effectively improved, but also the robustness of detecting the tobacco product to be detected in complex scenes is improved, and even in low light or occlusion conditions, it has a high accuracy.
[0067] In one embodiment, in order to make the obtained target image features and target point cloud features more accurate, as shown in FIG. Figure 3 As shown, an optional implementation in S201 includes:
[0068] S301: Acquire initial image information and initial point cloud data of a tobacco product to be inspected.
[0069] Optionally, in this embodiment, the original image features of the tobacco product to be detected are obtained based on a camera. The original image features are preprocessed to obtain initial image information of the tobacco product to be detected. Optionally, in this embodiment, the method for preprocessing the original image features is to perform denoising and enhancement processing on the original image features to obtain a first image feature. The first image feature is processed using an image enhancement tool provided by the algorithm to process an image in a low-light environment to obtain a second image feature. The image histogram equalization technology is used to enhance the image contrast of the second image feature to obtain a third image feature. The third image feature is subjected to image standardization processing to scale the pixel value of the third image feature to between 0 and 1 to obtain an initial image feature.
[0070] Optionally, this embodiment uses a laser radar to obtain raw point cloud features of the tobacco product to be inspected. The raw point cloud features are preprocessed to obtain initial point cloud data of the tobacco product to be inspected. This embodiment preprocesses the raw point cloud features by filtering the raw point cloud data to remove noise points, thereby obtaining a first point cloud feature. A voxel filter is then used to reduce the number of points in the first point cloud feature to obtain a second point cloud feature. The point cloud coordinates in the second point cloud feature are then converted from the laser radar coordinate system to the world coordinate system to obtain an initial point cloud feature.
[0071] S302: extracting and processing the initial image information based on a convolutional neural network to obtain target image features of the tobacco product to be detected.
[0072] Optionally, in this embodiment, the initial image information is extracted and processed based on a convolutional neural network to obtain target image features of the tobacco product to be inspected. In this embodiment, the convolutional neural network is pre-trained using the SMOKE model. The SMOKE model utilizes a convolutional neural network to extract deep features from the image, capturing detailed information and spatial relationships within the image. The extracted features include visual information such as object edges and texture. Based on these extracted features, extracted image features are determined, and the accuracy of the convolutional neural network output is determined. The relevant parameters of the convolutional neural network are then optimized to achieve the purpose of training the convolutional neural network.
[0073] S303: extracting and processing the initial point cloud data based on the point cloud extraction model to obtain target point cloud features of the tobacco product to be detected.
[0074] Optionally, in this embodiment, the initial point cloud data is extracted and processed based on the sparse convolutional layer in the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected. The point cloud extraction model can be a PointPillars model or a PointNet++ model.
[0075] In this embodiment, initial image information and initial point cloud data of the tobacco product to be inspected are obtained. The initial image information is extracted and processed using a convolutional neural network to obtain target image features of the tobacco product to be inspected. The initial point cloud data is then extracted and processed using a point cloud extraction model to obtain target point cloud features of the tobacco product to be inspected. This embodiment not only improves the efficiency of acquiring target image features and target point cloud features, but also improves their accuracy, thereby improving the accuracy of target detection results.
[0076] In one embodiment, in order to make the target detection result determined to be more accurate, such as Figure 4 As shown, an optional implementation of S203 includes:
[0077] S401: Input the fused features into the target detection model to obtain the initial detection results of the tobacco product to be detected.
[0078] Optionally, in this embodiment, the target detection model may adopt a PV-RCNN (Point-Voxel Feature Set Abstraction Network) model.
[0079] Optionally, the object detection model needs to be pre-trained. The training method is as follows: Within the algorithm framework, the stochastic gradient descent (SGD) optimization algorithm is used for training. The training data includes annotated images and point cloud data, where the annotations include object category, location, and size. During training, VisualDL is used to visualize the training process and monitor metrics such as the training loss function and accuracy in real time to ensure that the model converges and achieves the expected results. After training, the application needs to be evaluated. The evaluation method is as follows: the model is evaluated on a reserved test set using metrics such as AP (Average Precision), mAP (Mean Average Precision), and inference speed. Finally, a weighted sum of the AP, mAP, and inference speed is performed to obtain the summed result. If the summed result meets the judgment threshold, the object detection model is considered qualified for use.
[0080] S402, determining whether a first defect exists in the initial detection result.
[0081] Optionally, the initial detection result in this embodiment includes the position information, shape and defect information of the tobacco product to be detected, wherein the defect information includes the defect type and defect location.
[0082] Optionally, in this embodiment, defect information is obtained from the initial detection result. Based on the defect information, the presence of the first defect is determined. Specifically, if the defect information is available, the first defect presence is determined to be a defect. If the defect information is not available, the first defect presence is determined to be a defect.
[0083] S403 : Determine a target detection result of the tobacco product to be detected according to the existence of the first defect, the target image feature, and the initial detection result.
[0084] As an optional implementation of the embodiment of the present application, when the first defect existence condition is a defect, the defect type of the tobacco product to be inspected is determined, and the target detection result of the tobacco product to be inspected is determined based on the defect type, target image features, and initial detection results. In this embodiment, an optional implementation method for determining the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results is: when the defect type is a non-deformation defect, the existence of a second defect of the tobacco product to be inspected is determined based on the target image features; and when the second defect existence is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected; specifically, the target image features are input into the target detection model, and the target detection model determines the existence of the second defect in the target image features; and when the second defect existence is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected; when the second defect existence is no defect, a fourth detection result is determined based on the target image features and the candidate detection model (the target image features are input into the candidate detection model), and the fourth detection result is used as the target detection result; wherein the candidate detection model refers to a model that is pre-ranked according to the accuracy of the detection results of each original detection model, and the original detection model with the highest accuracy is used as the target detection model, and the original detection model ranked second is used as the candidate detection model. In the case where the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and in the case where the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected. In this embodiment, an optional implementation method for determining the existence of the third defect of the tobacco product to be inspected based on the target image features and the target point cloud features is to input the target image features into the target detection model, and the target detection model outputs the existence of the fourth defect. The target point cloud features are input into the target detection model, and the target detection model outputs the existence of the fifth defect. Based on the fourth defect existence and the fifth defect existence, the existence of the third defect is determined. Specifically, when the fourth defect existence and the fifth defect existence are both defects, the third defect existence is defects. When the fourth defect existence and the fifth defect existence are both defects, the third defect existence is defects.
[0085] As another optional implementation of the embodiment of the present application, when the first defect existence condition is that there is no defect, the target detection result of the tobacco product to be detected is determined based on the target image feature and the initial detection result. Specifically, the target image feature is input into the target detection model to obtain a third detection feature. The target detection result is determined based on the initial detection result and the third detection result. Exemplarily, if the initial detection result and the third detection result both include the position information of the tobacco product to be detected, the intermediate position information between the position information of the tobacco product to be detected contained in the first detection result and the position information of the tobacco product to be detected contained in the second detection result is used as the position information of the tobacco product to be detected in the target detection product.
[0086] In this embodiment, the fused features are input into the target detection model to obtain an initial detection result for the tobacco product to be inspected. The presence of a first defect in the initial detection result is determined. Based on the presence of the first defect, the target image features, and the initial detection result, a target detection result for the tobacco product to be inspected is determined. This embodiment effectively improves the accuracy of target detection results.
[0087] In one embodiment, Figure 5 As shown, an optional implementation of the tobacco product detection method is:
[0088] S501: Acquire initial image information and initial point cloud data of a tobacco product to be detected.
[0089] S502: extracting and processing the initial image information based on a convolutional neural network to obtain target image features of the tobacco product to be detected.
[0090] S503 : extracting and processing the initial point cloud data based on the point cloud extraction model to obtain target point cloud features of the tobacco product to be detected.
[0091] S504 , based on the middle layer of the neural network model, the target image features and the target point cloud features are fused to obtain fused features of the tobacco product to be detected.
[0092] S505: Input the fused features into the target detection model to obtain an initial detection result of the tobacco product to be detected.
[0093] S506: Determine whether the first defect exists in the initial detection result.
[0094] S507 : When the first defect existence condition is that a defect exists, determine the defect type of the tobacco product to be inspected.
[0095] S508, when the defect type is a non-deformation defect, determine the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the second defect presence is a defect, use the initial inspection result as the target inspection result of the tobacco product to be inspected.
[0096] S509, when the defect type is a deformation defect, determine the existence of a third defect of the tobacco product to be inspected based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, use the initial detection result as the target detection result of the tobacco product to be inspected.
[0097] The present application obtains the target image features and target point cloud features of the tobacco product to be detected. The target image features and target point cloud features are fused to obtain the fused features of the tobacco product to be detected. Based on the fused features and target image features, the target detection results of the tobacco product to be detected are determined. The present application obtains the fused features of the tobacco product to be detected by fusion processing of the target image features and target point cloud features, and determines the target detection results of the tobacco product to be detected based on the fused features. Through the fusion of multimodal data, not only the accuracy of the target detection results for the tobacco product to be detected is effectively improved, but also the robustness of detecting the tobacco product to be detected in complex scenes is improved, and it has high accuracy even in low light or occlusion conditions.
[0098] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.
[0099] Based on the same inventive concept, embodiments of the present application also provide a tobacco product detection device for implementing the aforementioned tobacco product detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more tobacco product detection device embodiments provided below can be found in the above-described limitations of the tobacco product detection method and will not be further elaborated here.
[0100] In one embodiment, Figure 6As shown, a tobacco product detection device 1 is provided, comprising: an acquisition module 10, a first determination module 20 and a second determination module 30, wherein:
[0101] An acquisition module 10 is used to acquire target image features and target point cloud features of the tobacco product to be detected;
[0102] The first determination module 20 is used to fuse the target image features and the target point cloud features to obtain the fused features of the tobacco product to be detected;
[0103] The second determination module 30 is configured to determine a target detection result of the tobacco product to be detected based on the fusion feature and the target image feature.
[0104] In one embodiment, the Figure 6 The acquisition module 10 is further specifically configured to:
[0105] Obtaining initial image information and initial point cloud data of the tobacco product to be inspected;
[0106] The initial image information is extracted and processed based on a convolutional neural network to obtain the target image features of the tobacco product to be detected;
[0107] The initial point cloud data is extracted and processed based on the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected.
[0108] In one embodiment, the Figure 6 The first determining module 20 is further specifically configured to:
[0109] Based on the middle layer of the neural network model, the target image features and the target point cloud features are fused to obtain the fused features of the tobacco products to be detected.
[0110] In one embodiment, the Figure 6 The second determining module 30 is further specifically configured to:
[0111] Input the fused features into the target detection model to obtain the initial detection results of the tobacco products to be detected;
[0112] Determining the presence of a first defect in the initial inspection result;
[0113] A target detection result of the tobacco product to be detected is determined according to the presence of the first defect, the target image feature and the initial detection result.
[0114] In one embodiment, the Figure 6 The second determining module 30 is further specifically configured to:
[0115] When the first defect existence condition is that a defect exists, determining the defect type of the tobacco product to be inspected;
[0116] Determine the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results.
[0117] In one embodiment, the Figure 6 The second determining module 30 is further specifically configured to:
[0118] When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the presence of the second defect is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected;
[0119] When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
[0120] Each module in the tobacco product detection device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.
[0121] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data on tobacco product detection. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a tobacco product detection method is implemented.
[0122] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0123] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0124] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0125] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0126] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0127] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: obtaining target image features and target point cloud features of the tobacco product to be detected, including:
[0128] Obtaining initial image information and initial point cloud data of the tobacco product to be inspected;
[0129] The initial image information is extracted and processed based on a convolutional neural network to obtain the target image features of the tobacco product to be detected;
[0130] The initial point cloud data is extracted and processed based on the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected.
[0131] In one embodiment, when executing the computer program, the processor further implements the following steps: fusing the target image features and the target point cloud features to obtain fused features of the tobacco product to be detected, including:
[0132] Based on the middle layer of the neural network model, the target image features and the target point cloud features are fused to obtain the fused features of the tobacco products to be detected.
[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: determining a target detection result of the tobacco product to be detected based on the fusion features, including:
[0134] Input the fused features into the target detection model to obtain the initial detection results of the tobacco products to be detected;
[0135] Determining the presence of a first defect in the initial inspection result;
[0136] A target detection result of the tobacco product to be detected is determined according to the presence of the first defect, the target image feature and the initial detection result.
[0137] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: determining a target detection result of the tobacco product to be inspected based on the presence of the first defect, the target image features, and the initial detection result, including:
[0138] When the first defect existence condition is that a defect exists, determining the defect type of the tobacco product to be inspected;
[0139] Determine the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results.
[0140] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: determining a target detection result of the tobacco product to be inspected based on the defect type, the target image feature, and the initial detection result, including:
[0141] When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the presence of the second defect is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected;
[0142] When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:
[0144] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0145] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0146] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0147] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtaining target image features and target point cloud features of the tobacco product to be detected, including:
[0148] Obtaining initial image information and initial point cloud data of the tobacco product to be inspected;
[0149] The initial image information is extracted and processed based on a convolutional neural network to obtain the target image features of the tobacco product to be detected;
[0150] The initial point cloud data is extracted and processed based on the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected.
[0151] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: fusing the target image features and the target point cloud features to obtain fused features of the tobacco product to be detected, including:
[0152] Based on the middle layer of the neural network model, the target image features and the target point cloud features are fused to obtain the fused features of the tobacco products to be detected.
[0153] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be detected based on the fusion features, including:
[0154] Input the fused features into the target detection model to obtain the initial detection results of the tobacco products to be detected;
[0155] Determining the presence of a first defect in the initial inspection result;
[0156] A target detection result of the tobacco product to be detected is determined according to the presence of the first defect, the target image feature and the initial detection result.
[0157] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be inspected based on the presence of the first defect, the target image features, and the initial detection result, including:
[0158] When the first defect existence condition is that a defect exists, determining the defect type of the tobacco product to be inspected;
[0159] Determine the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results.
[0160] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be inspected based on the defect type, the target image feature, and the initial detection result, including:
[0161] When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the presence of the second defect is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected;
[0162] When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
[0163] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0164] Obtaining target image features and target point cloud features of the tobacco product to be detected;
[0165] The target image features and the target point cloud features are fused to obtain the fused features of the tobacco product to be detected;
[0166] The target detection result of the tobacco product to be detected is determined based on the fusion features and the target image features.
[0167] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtaining target image features and target point cloud features of the tobacco product to be detected, including:
[0168] Obtaining initial image information and initial point cloud data of the tobacco product to be inspected;
[0169] The initial image information is extracted and processed based on a convolutional neural network to obtain the target image features of the tobacco product to be detected;
[0170] The initial point cloud data is extracted and processed based on the point cloud extraction model to obtain the target point cloud features of the tobacco product to be detected.
[0171] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: fusing the target image features and the target point cloud features to obtain fused features of the tobacco product to be detected, including:
[0172] Based on the middle layer of the neural network model, the target image features and the target point cloud features are fused to obtain the fused features of the tobacco products to be detected.
[0173] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be detected based on the fusion features, including:
[0174] Input the fused features into the target detection model to obtain the initial detection results of the tobacco products to be detected;
[0175] Determining the presence of a first defect in the initial inspection result;
[0176] A target detection result of the tobacco product to be detected is determined according to the presence of the first defect, the target image feature and the initial detection result.
[0177] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be inspected based on the presence of the first defect, the target image features, and the initial detection result, including:
[0178] When the first defect existence condition is that a defect exists, determining the defect type of the tobacco product to be inspected;
[0179] Determine the target detection result of the tobacco product to be inspected based on the defect type, target image features and initial detection results.
[0180] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining a target detection result of the tobacco product to be inspected based on the defect type, the target image feature, and the initial detection result, including:
[0181] When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image features; and when the presence of the second defect is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected;
[0182] When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
[0183] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.
[0184] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0185] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A tobacco product detection method, characterized in that: The method comprises: Obtaining target image features and target point cloud features of the tobacco product to be detected; fusing the target image features and the target point cloud features to obtain fused features of the tobacco product to be detected; A target detection result of the tobacco product to be detected is determined based on the fusion feature and the target image feature.
2. The method according to claim 1, characterized in that The step of obtaining target image features and target point cloud features of the tobacco product to be detected includes: Acquiring initial image information and initial point cloud data of the tobacco product to be detected; Extracting and processing the initial image information based on a convolutional neural network to obtain target image features of the tobacco product to be detected; The initial point cloud data is extracted and processed based on a point cloud extraction model to obtain target point cloud features of the tobacco product to be detected.
3. The method according to claim 1, characterized in that The fusing the target image features and the target point cloud features to obtain the fused features of the tobacco product to be detected includes: Based on a neural network model, the target image features and the target point cloud features are fused to obtain fused features of the tobacco product to be detected.
4. The method according to claim 1, wherein Determining the target detection result of the tobacco product to be detected based on the fusion feature includes: Inputting the fused features into a target detection model to obtain an initial detection result of the tobacco product to be detected; Determining the presence of a first defect in the initial detection result; A target detection result of the tobacco product to be detected is determined according to the existence of the first defect, target image features and the initial detection result.
5. The method according to claim 4, characterized in that The determining, based on the first defect presence, target image features, and the initial detection result, a target detection result of the tobacco product to be detected includes: When the first defect existence condition is that a defect exists, determining a defect type of the tobacco product to be inspected; A target detection result of the tobacco product to be detected is determined according to the defect type, the target image feature and the initial detection result.
6. The method according to claim 5, characterized in that Determining the target detection result of the tobacco product to be inspected according to the defect type, the target image feature, and the initial detection result includes: When the defect type is a non-deformation defect, determining the presence of a second defect of the tobacco product to be inspected based on the target image feature; and when the second defect presence is a defect, using the initial inspection result as the target inspection result of the tobacco product to be inspected; When the defect type is a deformation defect, the existence of the third defect of the tobacco product to be inspected is determined based on the target image features and the target point cloud features; and when the existence of the third defect is a defect, the initial detection result is used as the target detection result of the tobacco product to be inspected.
7. A tobacco product detection device, characterized in that: include: An acquisition module, used to acquire target image features and target point cloud features of the tobacco product to be detected; A first determination module is configured to fuse the target image features and the target point cloud features to obtain a fused feature of the tobacco product to be detected; The second determination module is configured to determine a target detection result of the tobacco product to be detected based on the fusion feature and the target image feature.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the tobacco product detection method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the tobacco product detection method according to any one of claims 1 to 6 are implemented.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the tobacco product detection method according to any one of claims 1 to 6 are implemented.