Multi-modal sensor fused method and system for dynamically detecting size of cut filter stick

Abstract

The embodiment of the invention provides a multi-modal sensor fused method and system for dynamically detecting the size of a cut filter stick, and belongs to the technical field of cigarette detection. The method comprises the following steps: acquiring real-time sensing data of a sensor group; correcting standard parameters according to the real-time sensing data; carrying out preprocessing operation on the corrected real-time sensing data; determining a fusion weight according to the real-time sensing data after the preprocessing operation; according to the fusion weight, the real-time sensing data and the standard parameters, a preset dynamic monitoring model is adopted for size detection. According to the method and the system, comprehensive analysis is carried out by fusing real-time sensing data of the sensor group, comprehensive detection is carried out on the size of the filter stick through multiple modes, and the accuracy of size detection of the filter stick is improved.

Description

Technical Field

[0001] This invention relates to the field of cigarette detection technology, and more specifically to a method and system for dynamic detection of filter rod dimensions after cutting using multimodal sensor fusion. Background Technology

[0002] As a crucial component of cigarettes, the dimensional accuracy of the filter rod after cutting (such as length, end face flatness, and axis perpendicularity) directly affects the cigarette rolling quality and appearance consistency. Existing post-cut filter rod inspection technologies have the following shortcomings: Detection limitations: Traditional photoelectric sensors can only detect whether the filter rod is missing, while visual inspection is easily affected by changes in lighting and differences in the color of the filter rod. Laser inspection has difficulty in identifying micro-defects on the end face (such as dents and burrs), resulting in low detection accuracy and high false negative rate. Poor dynamic adaptability: In high-speed cigarette machines (production speed of 10,000 cigarettes / minute), the filter rod is in a high-speed motion state, and the sampling frequency of a single sensor is insufficient, which can easily lead to dimensional measurement errors (above ±0.3mm) due to motion ambiguity. Delayed rejection response: The detection and rejection mechanisms lack closed-loop linkage, with a delay of more than 50ms from detection to rejection, resulting in non-conforming products not being accurately rejected and requiring secondary manual screening; Low data utilization: The detection data was not fused and analyzed in multiple dimensions, making it impossible to provide early warning of filter rod defects or provide data support for equipment parameter optimization. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for dynamic detection of the dimensions of a filter rod after cutting using multimodal sensor fusion, which enables accurate detection of the filter rod.

[0004] To achieve the above objectives, embodiments of the present invention provide a method for dynamic detection of the dimensions of a filter rod after cutting using multimodal sensor fusion, comprising: Acquire real-time sensing data from the sensor array; The standard parameters are corrected based on the real-time sensor data; Preprocess the corrected real-time sensor data; The fusion weights are determined based on the real-time sensor data after preprocessing. Size detection is performed using a preset dynamic monitoring model based on the fusion weights, the real-time sensing data, and the standard parameters.

[0005] Optionally, the real-time sensing data includes visual sensing information, laser acquisition information, and infrared acquisition information; The visual sensing information includes a visual image from the left side of the filter rod's travel direction, the laser acquisition information includes a laser image from the left side of the filter rod's travel direction, and the infrared acquisition information includes an infrared image from the right side of the filter rod's travel direction.

[0006] Optionally, real-time sensing data from the sensor array is acquired, including: Determine if the filter rod has reached the detection position; When the filter rod reaches the detection position, the sensor trigger time is calculated according to formula (1): (1) in, For sensor trigger time, The time it takes for the filter rod to reach the detection position. The unit is meters. The real-time speed of the filter rod. This is the preset minimum reserved time.

[0007] Optionally, real-time sensing data from the sensor array is acquired, including: Perform a Gaussian filter operation on the acquired visual image; Perform Canny edge detection on the result of Gaussian filtering; Perform a Hough transform operation on the results of the Canny edge detection operation; The acquired laser images were fitted with a cubic polynomial curve to obtain the filter rod tilt angle; Adaptive threshold segmentation is performed on the acquired infrared images; The area of ​​the heat-affected zone is calculated using the region growing method based on the results of the adaptive threshold segmentation operation.

[0008] Optionally, the standard parameters are corrected based on the real-time sensing data, including: Calculate the correction factor according to formula (2): (2) in, The correction coefficient is... The standard filter rod length before correction. The average filter rod length during the previous correction cycle. The standard filter rod tilt angle before correction. This represents the average filter rod tilt angle during the previous correction cycle. This represents the area of ​​the heat-affected zone before correction. This represents the average area of ​​the heat-affected zone during the previous correction period.

[0009] Optionally, determining the fusion weights based on the real-time sensing data includes: Modal consistency verification is performed on the real-time sensing data; The weight values ​​of the feature layer attention mechanism are adjusted based on the modality consistency test results; Update the feature layer attention mechanism.

[0010] Optionally, the feature layer attention mechanism is updated, including: (3) in, For the first The weight values ​​of each modality, For the first Modal accuracy of each modality For the standard deviation of other modes, The accuracy of the current modality. It is a modal set.

[0011] Optionally, size detection is performed using a preset dynamic monitoring model based on the fusion weights, the real-time sensing data, and the standard parameters, including: The fusion weights are used to perform feature fusion on the input real-time sensing data to obtain a fused feature vector; Establish state equations and observation equations based on the fused feature vectors; The physical quantities are mapped based on the state equation and the observation equation to obtain the detected size.

[0012] Optionally, size detection is performed using a preset dynamic monitoring model based on the fusion weights, the real-time sensing data, and the standard parameters, including: The detection threshold is determined according to formulas (4) and (5): (4) (5) in, This is the upper limit of the preset threshold range. This is the lower limit of the preset threshold range. This is the average length of the most recently preset number of qualified filter rods. Standard deviation For speed compensation coefficient, For the speed of the filter rod, For material compensation coefficient, These are the material parameters for the filter rod.

[0013] On the other hand, the present invention also provides a multimodal sensor fusion-based dynamic detection system for the dimensions of a filter rod after cutting, the system comprising: Sensor array, used to collect real-time sensing data; A processor for performing any of the methods described above.

[0014] Through the above technical solutions, the embodiments of the present invention provide a method and system for dynamic detection of filter rod dimensions after cutting using multimodal sensor fusion. The method and system comprehensively analyze the real-time sensing data of the fused sensor group and comprehensively detect the dimensions of the filter rod through multiple modes, thereby improving the accuracy of filter rod dimension detection.

[0015] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for dynamic detection of filter rod dimensions after cutting, based on a multimodal sensor fusion according to an embodiment of the present invention. Figure 2 This is a flowchart of a preprocessing operation method according to an embodiment of the present invention; Figure 3 This is a flowchart of a method for size detection using a preset dynamic monitoring model based on fusion weights, real-time sensing data, and standard parameters, according to an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0018] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0019] like Figure 1 The diagram shows a flowchart of a method for dynamic detection of the dimensions of a filter rod after cutting, based on multimodal sensor fusion according to an embodiment of the present invention. Figure 1 In this method, the steps may include: In step S10, real-time sensing data from the sensor group is acquired; In step S11, the standard parameters are corrected based on real-time sensing data; In step S12, the corrected real-time sensing data is preprocessed. In step S13, the fusion weights are determined based on the real-time sensing data after the preprocessing operation.

[0020] In step S14, size detection is performed using a preset dynamic monitoring model based on fusion weights, real-time sensing data, and standard parameters.

[0021] In such Figure 1 In the illustrated method, step S10 can be used to acquire real-time sensing data from the sensor array. This real-time sensing data can be in various forms known to those skilled in the art. In one example of the invention, the sensor data may include visual sensing information, laser acquisition information, and infrared acquisition information. Further, to ensure detection accuracy and avoid interference between sensor signals, the visual sensor (e.g., an industrial camera) used to acquire visual sensing information can be positioned on the left side of the filter rod production line's travel direction, the laser displacement sensor used to acquire laser acquisition information can be positioned on the left side of the filter rod production line's travel direction, and the infrared thermal imaging sensor used to acquire infrared acquisition information can be positioned on the right side of the filter rod production line's travel direction. Accordingly, the visual sensing information can be a visual image viewed from the left side of the filter rod's travel direction, the laser acquisition information can be a laser image viewed from the left side of the filter rod's travel direction, and the infrared acquisition information can be an infrared image viewed from the right side of the filter rod's travel direction.

[0022] Furthermore, the angle between the lens axis and the filter rod axis of the visual sensor used to acquire visual sensing information can be in the range of 42.5°±0.5°. The resolution of the visual sensor can be, for example, 2048×1080, and the frame rate can be, for example, 1000fps. During acquisition, the visual sensor can focus on the end face of the filter rod to acquire a contour image to extract the center coordinates and the percentage of rough edges. Further, in one example of the invention, the visual sensor can continuously capture at least two frames of end face images (contour images) at preset time intervals (e.g., 1.2ms).

[0023] The laser displacement sensor used to acquire laser acquisition information can have an angle of, for example, 30.5° between its emitting axis and the filter rod axis, a sampling frequency of, for example, 500kHz, and an accuracy range of ±0.01mm. During information acquisition, the laser displacement sensor can employ a three-segment Z-shaped scanning path to acquire 2000 sampling points to calculate the filter rod length and the filter rod tilt angle of its axis. Furthermore, in one example of the invention, the laser displacement sensor can acquire a preset number (e.g., 2000) of the outer circular coordinate points of the filter rod at a preset frequency (e.g., 500kHz).

[0024] The detection surface axis of the infrared thermal imaging sensor used to acquire infrared information can be perpendicular to the filter rod axis, and the resolution can be, for example, 640×512, with a thermal sensitivity range of <50mK. During information acquisition, the infrared thermal imaging sensor can focus on a 1.2mm annular area at the edge of the filter rod end face to acquire a temperature field image, thereby calculating the area of ​​the cut heat-affected zone. Furthermore, in one example of the invention, the infrared thermal imaging sensor can be used for a preset time length (80... Acquire at least one frame of temperature field image of the end face within the range.

[0025] Furthermore, considering that the filter rod is in motion on the production line, in order to avoid the high speed affecting the information acquisition of the filter rod, in one example of the present invention, when acquiring information, it can first be determined whether the filter rod has reached the detection position. If the filter rod has reached the detection position, the sensor trigger time is calculated according to the following formula (1): (1) in, For sensor trigger time, The time it takes for the filter rod to reach the detection position. The unit is meters. The real-time speed of the filter rod. This is the preset minimum reserved time.

[0026] Step S11 can be used to correct the standard parameter based on real-time sensing data, thereby avoiding inaccuracies in the real-time sensing data. In one example of the present invention, the method for correcting the standard parameter can be implemented by a coefficient product method. Specifically, the correction coefficient of the coefficient product can be calculated using the following formula (2): (2) in, For correction factor, The standard filter rod length before correction. The average filter rod length during the previous correction cycle. The standard filter rod tilt angle before correction. This represents the average filter rod tilt angle during the previous correction cycle. This represents the area of ​​the heat-affected zone before correction. This represents the average area of ​​the heat-affected zone during the previous correction period.

[0027] Step S12 can be used to preprocess the corrected real-time sensing data. The specific method for this preprocessing can be of various forms known to those skilled in the art. In one example of the present invention, step S10 may further include, for example... Figure 2 The method shown. Specifically, in this Figure 2 In this process, step S10 may further include the following steps: In step S20, a Gaussian filter operation is performed on the acquired visual image. Specifically, this Gaussian filter operation... The value can be 1.2.

[0028] In step S21, a Canny edge detection operation is performed on the result of the Gaussian filtering operation. Specifically, in this example, the threshold value of the Canny edge detection operation can be, for example, 50-150.

[0029] In step S22, a Hough transform operation is performed on the results of the Canny edge detection operation. Specifically, in this example, the Hough transform operation may first extract the center coordinates of the circle, determine the percentage of rough edges based on the center coordinates, and finally use the percentage of rough edges to make a threshold judgment (e.g., >4.5% is marked as potentially unqualified) for screening and filtering.

[0030] In step S23, a cubic polynomial curve fitting operation is performed on the acquired laser image to obtain the filter rod tilt angle. Specifically, in this example, the cubic polynomial curve fitting operation can first determine the length of the filter rod using the following formula (6): (6) in, For the calculated length, The number of sampling points. The angle between the laser emission axis and the filter rod axis. For the first Coordinates of each collection point For the first The coordinates of each collection point are then determined. Based on this length, the center deviation is determined, and based on this center deviation, the tilt angle of the filter rod is determined.

[0031] In step S24, an adaptive thresholding operation is performed on the acquired infrared image. Specifically, in this example, the adaptive thresholding operation may, for example, select a region with a temperature > 46°C as an approximate heat-affected zone.

[0032] In step S25, the area of ​​the heat-affected zone is calculated using the region growing method based on the result of the adaptive threshold segmentation operation. Specifically, in this example, the area of ​​the heat-affected zone can be calculated using the following formula (7): (7) in, The area of ​​the heat-affected zone. This is a set of pixels that approximate the heat-affected zone. As an identifier for the existence of pixels, This represents the pixel temperature (°C) in that region.

[0033] Step S13 can be used to determine the fusion weights based on the real-time sensing data after preprocessing. Specifically, in one embodiment of the present invention, the dimensions of the real-time sensing data after preprocessing can be, for example, visual features (2048 dimensions), laser features (128 dimensions), and infrared features (512 dimensions). Step S13 can first perform a modal consistency check on the real-time sensing data at the feature layer. If the consistency check result is poor, the modal weight of the corresponding feature can be reduced (the reduction value can be a preset value). More specifically, the modal consistency check can be performed by calculating the deviation between two modalities (e.g., the length deviation calculated from the laser feature and the visual feature). ), Then the consistency test result can be considered poor.

[0034] After correcting the weight values, step S13 can be performed by updating the feature layer attention mechanism according to the following formula (3): (3) in, For the first The weight values ​​of each modality, For the first Modal accuracy of each modality For the standard deviation of other modes, This represents the accuracy of the current modality (e.g., vision, laser). (The higher the accuracy, the more reliable the modality.) It is a modal set.

[0035] Step S14 can be used to perform size detection using a preset dynamic monitoring model based on fusion weights, real-time sensing data, and standard parameters. In this embodiment, step S14 may further include, for example: Figure 3 The method shown in the diagram. Figure 3 In this process, step S14 may further include the following methods: In step S30, a feature fusion operation is performed on the input real-time sensing data using fusion weights to obtain a fused feature vector. The value of the fused feature vector can be in the range of [0,1].

[0036] In step S31, the state equation and observation equation are established based on the fused feature vector. The state equation can be expressed as the following formula (8): (8) in, For the first Each state value It is the identity matrix. The characteristic state value of the previous time step. This is process noise.

[0037] The observation equation can be expressed as the following formula (9): (9) in, For the first One observation value, It is a diagonal matrix. To observe noise.

[0038] In step S32, physical quantities are mapped according to the state equation and the observation equation to obtain the detected size. In one example of the present invention, the filter rod length can be output using the following formula (10): (10) in, The length of the filter rod. This is the vector corresponding to the length of the filter rod in the fused feature vector; The tilt angle of the filter rod axis can be output using the following formula (11): (11) in, The angle of inclination of the filter rod. This is the vector corresponding to the tilt angle of the filter rod in the fused feature vector; The area of ​​the heat-affected zone can be output using the following formula (12): (12) in, This is the area of ​​the heat-affected zone. This is the vector corresponding to the area of ​​the heat-affected zone in the fused feature vector.

[0039] The specific method for size detection in step S14 can be of various forms known to those skilled in the art, including but not limited to detection methods based on preset intervals or preset thresholds. Considering the inherent instability of the device itself, in one example of the present invention, the size detection method can be determined using the following formulas (4) and (5): (4) (5) in, This is the upper limit of the preset threshold range. This is the lower limit of the preset threshold range. This is the average length of the most recently preset number of qualified filter rods. Standard deviation For speed compensation coefficient, For the speed of the filter rod, For material compensation coefficient, These are the material parameters for the filter rod.

[0040] For qualified products (within the calculated preset threshold) and unqualified products (outside the calculated preset threshold) in size inspection, in one example of the present invention, qualified products may be transported to other downstream coiling processes via a conveyor channel, while unqualified products may be rejected according to a preset delay rejection time. This delay rejection time may be one of several values ​​known to those skilled in the art. In one example of the present invention, the delay rejection time may be calculated using a method similar to formula (1) above, and the response time is set to a range of ≤50ms.

[0041] On the other hand, the present invention also provides a multimodal sensor fusion-based dynamic size detection system for filter rods after cutting. The system includes a sensor group and a processor, wherein the sensor group can be used to acquire real-time sensing data, and the processor can be used to perform actions such as... Figures 1 to 3 The method in the middle.

[0042] Specifically, in this embodiment, the sensor group may include multiple sensors. In one example of the present invention, the sensor group may include a vision sensor, a laser displacement sensor, and an infrared thermal imaging sensor. Further, the sensor group may adopt an asymmetrical layout of "left main side + right auxiliary side," with the left bracket integrating an industrial camera A and a laser displacement sensor A, and the right bracket integrating only an infrared thermal imaging sensor B, reducing signal interference between the two sides. That is, the vision sensor (e.g., an industrial camera) used to acquire visual sensing information can be located on the left side of the filter rod production line's travel direction, the laser displacement sensor used to acquire laser acquisition information can be located on the left side of the filter rod production line's travel direction, and the infrared thermal imaging sensor used to acquire infrared acquisition information can be located on the right side of the filter rod production line's travel direction. Correspondingly, the visual sensing information can be a visual image from the left perspective of the filter rod's travel direction, the laser acquisition information can be a laser image from the left perspective of the filter rod's travel direction, and the infrared acquisition information can be an infrared image from the right perspective of the filter rod's travel direction.

[0043] Furthermore, the angle between the lens axis and the filter rod axis of the visual sensor used to acquire visual sensing information can be in the range of 42.5°±0.5°. The resolution of the visual sensor can be, for example, 2048×1080, and the frame rate can be, for example, 1000fps. During acquisition, the visual sensor can focus on the end face of the filter rod to acquire a contour image to extract the center coordinates and the percentage of rough edges. Further, in one example of the invention, the visual sensor can continuously capture at least two frames of end face images (contour images) at preset time intervals (e.g., 1.2ms).

[0044] The laser displacement sensor used to acquire laser acquisition information can have an angle of, for example, 30.5° between its emitting axis and the filter rod axis, a sampling frequency of, for example, 500kHz, and an accuracy range of ±0.01mm. During information acquisition, the laser displacement sensor can employ a three-segment Z-shaped scanning path to acquire 2000 sampling points to calculate the filter rod length and the filter rod tilt angle of its axis. Furthermore, in one example of the invention, the laser displacement sensor can acquire a preset number (e.g., 2000) of the outer circular coordinate points of the filter rod at a preset frequency (e.g., 500kHz).

[0045] The detection surface axis of the infrared thermal imaging sensor used to acquire infrared information can be perpendicular to the filter rod axis, and the resolution can be, for example, 640×512, with a thermal sensitivity range of <50mK. During information acquisition, the infrared thermal imaging sensor can focus on a 1.2mm annular area at the edge of the filter rod end face to acquire a temperature field image, thereby calculating the area of ​​the cut heat-affected zone. Furthermore, in one example of the invention, the infrared thermal imaging sensor can be used for a preset time length (80... The processor acquires at least one frame of the temperature field image of the end face. The processor can be used to perform the following steps: In step S10, real-time sensing data from the sensor group is acquired; In step S11, the standard parameters are corrected based on real-time sensing data; In step S12, the corrected real-time sensing data is preprocessed. In step S13, the fusion weights are determined based on the real-time sensing data after the preprocessing operation.

[0046] In step S14, size detection is performed using a preset dynamic monitoring model based on fusion weights, real-time sensing data, and standard parameters.

[0047] Considering that the filter rod is in motion on the production line, in order to avoid the high speed affecting the information acquisition of the filter rod, in one example of the present invention, when acquiring information, it can first be determined whether the filter rod has reached the detection position. If the filter rod has reached the detection position, the sensor trigger time is calculated according to the following formula (1): (1) in, For sensor trigger time, The time it takes for the filter rod to reach the detection position. The unit is meters. The real-time speed of the filter rod. This is the preset minimum reserved time.

[0048] Step S11 can be used to correct the standard parameter based on real-time sensing data, thereby avoiding inaccuracies in the real-time sensing data. In one example of the present invention, the method for correcting the standard parameter can be implemented by a coefficient product method. Specifically, the correction coefficient of the coefficient product can be calculated using the following formula (2): (2) in, For correction factor, The standard filter rod length before correction. The average filter rod length during the previous correction cycle. The standard filter rod tilt angle before correction. This represents the average filter rod tilt angle during the previous correction cycle. This represents the area of ​​the heat-affected zone before correction. This represents the average area of ​​the heat-affected zone during the previous correction period.

[0049] Step S12 can be used to preprocess the corrected real-time sensing data. The specific method for this preprocessing can be of various forms known to those skilled in the art. In one example of the present invention, step S10 may further include, for example... Figure 2 The method shown. Specifically, in this Figure 2 In this process, step S10 may further include the following steps: In step S20, a Gaussian filter operation is performed on the acquired visual image. Specifically, this Gaussian filter operation... The value can be 1.2.

[0050] In step S21, a Canny edge detection operation is performed on the result of the Gaussian filtering operation. Specifically, in this example, the threshold value of the Canny edge detection operation can be, for example, 50-150.

[0051] In step S22, a Hough transform operation is performed on the results of the Canny edge detection operation. Specifically, in this example, the Hough transform operation may first extract the center coordinates of the circle, determine the percentage of rough edges based on the center coordinates, and finally use the percentage of rough edges to make a threshold judgment (e.g., >4.5% is marked as potentially unqualified) for screening and filtering.

[0052] In step S23, a cubic polynomial curve fitting operation is performed on the acquired laser image to obtain the filter rod tilt angle. Specifically, in this example, the cubic polynomial curve fitting operation can first determine the length of the filter rod using the following formula (6): (6) in, For the calculated length, Number of data collection points The angle between the laser emission axis and the filter rod axis. For the first Coordinates of each collection point For the first The coordinates of each collection point are then determined. Based on this length, the center deviation is determined, and based on this center deviation, the tilt angle of the filter rod is determined.

[0053] In step S24, an adaptive thresholding operation is performed on the acquired infrared image. Specifically, in this example, the adaptive thresholding operation may, for example, select a region with a temperature > 46°C as an approximate heat-affected zone.

[0054] In step S25, the area of ​​the heat-affected zone is calculated using the region growing method based on the result of the adaptive threshold segmentation operation. Specifically, in this example, the area of ​​the heat-affected zone can be calculated using the following formula (7): (7) in, The area of ​​the heat-affected zone. This is a set of pixels that approximate the heat-affected zone. As an identifier for the existence of pixels, This represents the pixel temperature (°C) in that region.

[0055] Step S13 can be used to determine the fusion weights based on the real-time sensing data after preprocessing. Specifically, in one embodiment of the present invention, the dimensions of the real-time sensing data after preprocessing can be, for example, visual features (2048 dimensions), laser features (128 dimensions), and infrared features (512 dimensions). Step S13 can first perform a modal consistency check on the real-time sensing data at the feature layer. If the consistency check result is poor, the modal weight of the corresponding feature can be reduced (the reduction value can be a preset value). More specifically, the modal consistency check can be performed by calculating the deviation between two modalities (e.g., the length deviation calculated from the laser feature and the visual feature). ), Then the consistency test result can be considered poor.

[0056] After correcting the weight values, step S13 can be performed by updating the feature layer attention mechanism according to the following formula (3): (3) in, For the first The weight values ​​of each modality, For the first Modal accuracy of each modality For the standard deviation of other modes, This represents the accuracy of the current modality (e.g., vision, laser). (The higher the accuracy, the more reliable the modality.) It is a modal set.

[0057] Step S14 can be used to perform size detection using a preset dynamic monitoring model based on fusion weights, real-time sensing data, and standard parameters. In this embodiment, step S14 may further include, for example: Figure 3 The method shown in the diagram. Figure 3 In this process, step S14 may further include the following methods: In step S30, a feature fusion operation is performed on the input real-time sensing data using fusion weights to obtain a fused feature vector. The value of the fused feature vector can be in the range of [0,1].

[0058] In step S31, the state equation and observation equation are established based on the fused feature vector. The state equation can be expressed as the following formula (8): (8) in, For the first Each state value It is the identity matrix. The characteristic state value of the previous time step. This is process noise.

[0059] The observation equation can be expressed as the following formula (9): (9) in, For the first One observation value, It is a diagonal matrix. To observe noise.

[0060] In step S32, physical quantities are mapped according to the state equation and the observation equation to obtain the detected size. In one example of the present invention, the filter rod length can be output using the following formula (10): (10) in, The length of the filter rod. This is the vector corresponding to the length of the filter rod in the fused feature vector; The tilt angle of the filter rod axis can be output using the following formula (11): (11) in, The angle of inclination of the filter rod. This is the vector corresponding to the tilt angle of the filter rod in the fused feature vector; The area of ​​the heat-affected zone can be output using the following formula (12): (12) in, This is the area of ​​the heat-affected zone. This is the vector corresponding to the area of ​​the heat-affected zone in the fused feature vector.

[0061] The specific method for size detection in step S14 can be of various forms known to those skilled in the art, including but not limited to detection methods based on preset intervals or preset thresholds. Considering the inherent instability of the device itself, in one example of the present invention, the size detection method can be determined using the following formulas (4) and (5): (4) (5) in, This is the upper limit of the preset threshold range. This is the lower limit of the preset threshold range. This is the average length of the most recently preset number of qualified filter rods. Standard deviation For speed compensation coefficient, For the speed of the filter rod, For material compensation coefficient, These are the material parameters for the filter rod.

[0062] For qualified products (within the calculated preset threshold) and unqualified products (outside the calculated preset threshold) in size inspection, in one example of the present invention, qualified products may be transported to other downstream coiling processes via a conveyor channel, while unqualified products may be rejected according to a preset delay rejection time. This delay rejection time may be one of several values ​​known to those skilled in the art. In one example of the present invention, the delay rejection time may be calculated using a method similar to formula (1) above, and the response time is set to a range of ≤50ms.

[0063] Through the above technical solutions, the embodiments of the present invention provide a method and system for dynamic detection of filter rod dimensions after cutting using multimodal sensor fusion. The method and system comprehensively analyze the real-time sensing data of the fused sensor group and comprehensively detect the dimensions of the filter rod through multiple modes, thereby improving the accuracy of filter rod dimension detection.

[0064] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0066] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0068] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0069] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0070] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0071] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0072] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for dynamic detection of the dimensions of a filter rod after cutting using multimodal sensor fusion, characterized in that, The method comprises the following steps: acquiring real-time sensing data of a sensor group; correcting a standard parameter according to the real-time sensing data; performing a preprocessing operation on the corrected real-time sensing data; determining a fusion weight according to the real-time sensing data after the preprocessing operation; performing size detection by using a preset dynamic monitoring model according to the fusion weight, the real-time sensing data, and the standard parameter.

2. The method of claim 1, wherein, The real-time sensing data comprises visual sensing information, laser collection information, and infrared collection information; The visual sensing information comprises a visual image of a left side view angle in a filter rod travel direction, the laser collection information comprises a laser image of the left side view angle in the filter rod travel direction, and the infrared collection information comprises an infrared image of a right side view angle in the filter rod travel direction.

3. The method of claim 2, wherein, The method of acquiring real-time sensing data of a sensor group comprises the following steps: determining whether a filter rod has reached a detection position; calculating a sensor trigger time according to formula (1) in the case that the filter rod has reached the detection position: ，（1） wherein, is the sensor trigger time, is the time of filter rod arrival at the detection position, is the unit meter, is the real-time speed of the filter rod, is the preset minimum reserved time.

4. The method of claim 2, wherein, The method of acquiring real-time sensing data of a sensor group comprises the following steps: performing a Gaussian filtering operation on the collected visual image; performing a Canny edge detection operation on the result of the Gaussian filtering operation; performing a Hough transform operation on the result of the Canny edge detection operation; performing a cubic polynomial curve fitting operation on the collected laser image to obtain a filter rod tilt angle; performing an adaptive threshold segmentation operation on the collected infrared image; calculating a heat-affected zone area according to the result of the adaptive threshold segmentation operation by using a region growing method.

5. The method of claim 1, wherein, The method of correcting a standard parameter according to the real-time sensing data comprises the following steps: calculating a correction coefficient according to formula (2): ，（2） wherein, is the correction factor, is the standard filter rod length before correction, is the average filter rod length in the previous correction period, is the standard filter rod tilt angle before correction, is the average filter rod tilt angle in the previous correction period, is the heat affected zone area before correction, is the average heat affected zone area in the previous correction period.

6. The method of claim 1, wherein, The method of determining a fusion weight according to the real-time sensing data comprises the following steps: performing a modal consistency test on the real-time sensing data; correcting a weight value of a feature layer attention mechanism according to the result of the modal consistency test; updating the feature layer attention mechanism.

7. The method of claim 6, wherein, The method of updating the feature layer attention mechanism comprises the following steps: ，（3） wherein, is a weight value for the th modality, is a modality accuracy for the th modality, is a standard deviation for other modalities, is an accuracy for a current modality, is a set of modalities.

8. The method of claim 1, wherein, The method of performing size detection by using a preset dynamic monitoring model according to the fusion weight, the real-time sensing data, and the standard parameter comprises the following steps: performing a feature fusion operation on the input real-time sensing data by using the fusion weight to obtain a fusion feature vector; establishing a state equation and an observation equation according to the fusion feature vector; performing physical quantity mapping according to the state equation and the observation equation to obtain a detected size.

9. The method of claim 1, wherein, The method of performing size detection by using a preset dynamic monitoring model according to the fusion weight, the real-time sensing data, and the standard parameter comprises the following steps: determining a detection threshold according to formula (4) and formula (5): ，（4） ，（5） wherein, is an upper limit of a preset threshold interval, is a lower limit of a preset threshold interval, is a mean value of lengths of a most recent preset number of filter rods that passed inspection, is a standard deviation, is a speed compensation coefficient, is a speed of the filter rod, is a material compensation coefficient, is a filter rod material parameter.

10. A multi-modal sensor fusion dynamic cut rod size detection system, characterized in that, The system comprises: a sensor group for collecting real-time sensing data; a processor for performing the method according to any one of claims 1 to 9.

Images