Method and system for automatically sorting and rechecking whole pack of instant lottery tickets based on super-resolution vision

Abstract

The invention provides an automatic sorting and rechecking method and system for a whole pack of instant lottery tickets based on super-resolution vision, and relates to the technical field of machine vision. Details of motion blur loss are recovered through dynamic inter-frame alignment and multi-scale feature fusion, and a high-resolution image is generated. Static visual features are extracted, lottery ticket scrape-open layer film fluorescence spectrum response features of the first page of the lottery ticket bag are obtained, film material distribution characteristics are obtained through spectrum decomposition and fused into multi-modal features, the multi-modal features are matched with a standard library, and a preliminary sorting result is obtained. In the sorting process, the grabbing time sequence consistency of the mechanical arm is monitored, re-checking is started and abnormal time period characteristics are backtracked if the result reaches the threshold critical value or the time sequence is abnormal, and a re-checking report is formed if a plurality of judgment results are not consistent, so that the accuracy and reliability of automatic sorting of the whole lottery ticket can be guaranteed.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to an automatic sorting and verification method, system, electronic device and storage medium for packaged instant lottery tickets based on super-resolution vision. Background Technology

[0002] The operations center faces a key constraint in the sorting and verification process of packaged instant lottery tickets: the lottery tickets are pre-sealed by the lottery center and unsealing is strictly prohibited, therefore non-contact detection technology must be used. The sorting system needs to accurately identify the type, batch, and authenticity of lottery tickets in a high-speed assembly line environment (typically ≥5 packages / second), while ensuring that packaging wear and stains caused by transportation or storage do not lead to misjudgments. In addition, the industry requires high sorting accuracy and real-time verification and traceability capabilities for critical sorting states (such as blurred features, weak signals) and mechanical operation anomalies (such as gripping deviation, timing errors). Traditional methods relying on manual visual inspection or single sensors can no longer meet the business requirements of high throughput and zero unsealing.

[0003] Currently, lottery ticket sorting primarily relies on static image acquisition systems using high-speed industrial cameras. These systems capture images of visible markings such as barcodes and batch numbers printed on the outside of the lottery tickets and then perform pattern matching. In terms of image processing, most systems use traditional image enhancement algorithms (such as histogram equalization and Wiener filtering) to mitigate motion blur and feature point matching (such as SIFT and ORB) to locate and extract the marked areas. The mechanical control layer depends on closed-loop control of the encoder's feedback position signal and a preset gripping trajectory to ensure the execution of the sorting action.

[0004] However, image blurring caused by high-speed motion is difficult to eliminate effectively using traditional image enhancement algorithms, especially when there is slight wear or reflection on the surface of the lottery bag printing, the error rate of identification increases significantly. Furthermore, existing mechanical control strategies rely solely on position sensor feedback and cannot perceive the temporal coordination between the grasping action and the visual recognition result. They lack a dynamic verification mechanism for critical sorting threshold situations (such as feature matching confidence being at the tolerance boundary) or mechanical anomalies (such as grasping delay or vibration offset), resulting in missorted bags requiring secondary manual inspection, which severely restricts sorting efficiency and automation levels. Summary of the Invention

[0005] The purpose of this application is to provide an automatic sorting and verification method, system, electronic device and storage medium for packaged instant lottery tickets based on super-resolution vision, so as to solve the problem of low verification and sorting accuracy in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides an automatic sorting and verification method for whole packages of instant lottery tickets based on super-resolution vision, comprising: Acquire clear, continuous video stream images of lottery packages in high-speed motion; Super-resolution reconstruction processing is performed on the continuous video stream images. Through dynamic inter-frame alignment and multi-scale feature fusion, the detailed information lost on the surface of the lottery package due to motion blur is restored to generate a high-resolution image sequence. Static visual features of the lottery package surface are extracted from the high-resolution image sequence. Simultaneously, the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package are obtained. The fluorescence spectral response features are subjected to spectral decomposition to obtain the distribution characteristics of the coating material of the first lottery ticket. The static visual features and the distribution characteristics of the coating material are combined to form a multimodal fusion feature. The fusion feature is matched and compared with a pre-stored standard feature library to obtain preliminary sorting results. During the sorting process, the timing consistency of the robotic arm's gripping actions is continuously monitored. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated. The system traces back the fusion features of lottery packages within a specific time period before and after the occurrence of the anomaly. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the system combines the anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

[0007] Optionally, after performing super-resolution reconstruction processing on the continuous video stream images, and recovering the detail information lost on the surface of the lottery package due to motion blur through dynamic inter-frame alignment and multi-scale feature fusion to generate a high-resolution image sequence, the method further includes: Obtain the optical properties of the transparent heat-shrink film used for lottery ticket packaging, including its transmittance distribution characteristics and surface refractive properties. The method further includes: By combining the static visual features, the distribution characteristics of the coating material of the first lottery ticket, and the optical properties of the transparent heat-shrink film, a multimodal fusion feature is formed. The multimodal fusion features are matched and compared with a pre-stored standard feature library. The quality of the entire package of lottery tickets is judged based on the coating status of the first lottery ticket and the integrity of the outer packaging, and a preliminary sorting result is obtained.

[0008] Optionally, the fluorescence spectral response characteristics of the scratch-off layer coating corresponding to the lottery ticket on the homepage of the lottery package are obtained. The fluorescence spectral response characteristics are then subjected to spectral decomposition to obtain the distribution characteristics of the coating material of the first lottery ticket. These are combined with the static visual characteristics and the distribution characteristics of the coating material to form a multimodal fusion feature, including: Based on the positioning information of the high-resolution image sequence, the multispectral optical imaging system is controlled to excite the scratch-off layer coating of the first lottery ticket using multiple light sources of different wavelengths in a time-division polling manner. The multispectral optical imaging system is used to acquire fluorescence images of the excited scraped layer in multiple spectral bands, and a continuous spectral reflectance curve for each pixel is generated based on the fluorescence images. The spectral reflectance curve is decomposed into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum, and the contribution of each spectral component to the pixel is calculated. Based on the contribution analysis, the continuous variation characteristics of the standard spectrum of the coating in spatial distribution are analyzed to identify whether there are uneven thickness or abnormal coverage areas in the scratch-off layer of the first lottery ticket as the coating anomaly identification result. At the same time, based on the temporal variation of multiple frames in the high-resolution image sequence, the motion posture change characteristics of the lottery package are analyzed. The static visual features, the results of the film overlay anomaly recognition, and the motion posture change features are fused in a multimodal manner to form a fused feature.

[0009] Optionally, the spectral reflectance curve is decomposed into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum, and the contribution of each spectral component to the pixel is calculated, including: For each pixel of the scratch-off layer of the first lottery ticket in the lottery package, a corresponding spectral reflectance curve is established. The spectral reflectance curve is composed of the reflectance intensity values ​​of multiple spectral bands and is expressed as a weighted sum of the corresponding components of the standard spectral of the coating, the corresponding components of the spectral of the paper substrate, and the corresponding components of the spectral of the printing ink. By minimizing the difference between the spectral reflectance curve and the weighted sum, the weighting coefficients of the corresponding components of the coated standard spectrum, the corresponding components of the paper substrate spectrum, and the corresponding components of the printing ink spectrum are solved. The weighting coefficients are used as the contribution of each spectral component to the corresponding pixel.

[0010] Optionally, based on the contribution analysis of the continuous variation characteristics of the standard spectrum of the coating in spatial distribution, the presence of uneven thickness or abnormal coverage areas in the scratched layer is identified as the coating anomaly identification result. Simultaneously, based on the temporal changes of multiple frames in the high-resolution image sequence, the motion posture change characteristics of the lottery bag are analyzed, including: The contribution of the corresponding component of the coating standard spectrum in each pixel is arranged according to its actual position. The change of contribution of adjacent pixels is detected and a continuous numerical sequence is recorded. The spatial continuity variation characteristics of the coating standard spectrum are determined by the fluctuation amplitude and change frequency of the continuous numerical sequence. Based on the continuous change characteristics, regions where the contribution of multiple consecutive pixels changes beyond a preset range and where the overall contribution of the region differs significantly from the surrounding area are identified as areas with uneven thickness of the scraping layer. The region with a local contribution value of zero or far exceeding the normal range, and which forms an abrupt boundary with the surrounding area, is identified as an abnormal coverage area of ​​the scraped layer. The location information of the uneven thickness area and the abnormal coverage area are integrated to form the film abnormality identification result. Fixed markers on the surface of the lottery bag are selected from the high-resolution image sequence, and the coordinate information of the fixed markers on the surface of the lottery bag is obtained by tracking the position changes of the markers in each frame of the image. Based on the coordinate information, the relative change of the coordinates of the marker points in adjacent frames is calculated, and the rotation angle and displacement distance of the lottery package per unit time are determined based on the relative change. According to the change law of the rotation angle and displacement distance of the lottery package, the motion posture change characteristics of the lottery package are formed.

[0011] Optionally, the detailed information includes category identification and micro-anti-counterfeiting patterns; the super-resolution reconstruction processing of the continuous video stream images, through dynamic inter-frame alignment and multi-scale feature fusion, restores the detailed information lost on the surface of the lottery package due to motion blur, in order to generate a high-resolution image sequence, includes: Multiple temporally consecutive frames are selected from the continuous video stream as input frames. The scale-invariant feature transform algorithm is used to extract feature points of each frame in the input frame group, and the fast nearest neighbor search matching algorithm is used to calculate the correspondence between the feature points of different frames. Based on the feature points and their correspondence, the positional offset between each pair of feature points is calculated, and an affine transformation algorithm is used to perform geometric transformation on each frame image in the input frame group based on the positional offset, so that the relative positions of the lottery package in multiple frames remain consistent. The grayscale values ​​of each pixel in the aligned multi-frame images are weighted and averaged to generate a preliminary fused image. The weights of the weighted average calculation are dynamically adjusted according to the signal-to-noise ratio of each pixel in different frames. The preliminary fused image is input into a multi-layer processing structure, and multi-scale features are extracted through convolution operations to obtain feature maps containing different levels of abstraction. The feature map is upsampled to increase its resolution and gradually restore spatial details. By using skip connections, low-level detailed features are fused with upsampled high-level semantic features to generate enhanced feature representations; Based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by dynamic trailing are reconstructed layer by layer, the text outline of the category identification and the detailed features of the micro anti-counterfeiting pattern are restored, and a high-resolution image sequence is output.

[0012] Optionally, based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by dynamic trailing are reconstructed layer by layer, restoring the text outline of the category identifier and the detailed features of the micro-anti-counterfeiting pattern, and outputting a high-resolution image sequence, including: The basic features of edge texture and the detailed features of high-frequency information are separated from the enhanced feature representation; Based on the basic features of the edge texture, in the area covered by the dynamic trailing shadow, missing edge segments and broken contour lines on the surface of the lottery package are supplemented, and the detailed features of the high-frequency information are filled in within the edge texture framework to restore the texture's light and dark transitions and subtle undulations. Based on the text-related feature components in the enhanced feature representation, the contrast of the stroke edges of the characters is strengthened and broken parts are repaired in the category identification area to form a continuous and closed character outline. Based on the feature points with a size smaller than a preset threshold in the enhanced feature representation, adjacent feature points are connected in the miniature anti-counterfeiting pattern area according to the pattern arrangement rule to form a complete pattern unit and supplement the connection structure between units. The restored edge texture, high-frequency information, category identification text outline, and micro anti-counterfeiting pattern are integrated into a single high-resolution image. All the generated single high-resolution images are arranged in chronological order to form a high-resolution image sequence.

[0013] Optionally, the fusion features of lottery packages within a specific time period before and after the anomaly are retrieved. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the anomaly type judgment result is generated and a review report is formed by combining the anomaly timestamp and workstation location information, including: The fusion features of lottery packages within a specific time period before and after the anomaly occurred are traced back and analyzed based on the static visual features and the distribution characteristics of the coating material in the fusion features, respectively, to obtain the judgment results of the static visual features and the judgment results based on the distribution characteristics of the coating material. The judgment results of the static visual features are compared with the judgment results based on the distribution characteristics of the coating material. When the two are inconsistent, they are marked as feature judgment conflict events. The grabbing anomaly timestamp and the workstation location information of the robot arm when the feature judgment conflict event occurs are obtained. Based on the aforementioned features, the type of conflict event is determined, abnormal timestamps and workstation location information are captured, an abnormality type determination result containing the cause of the abnormality, the location and time of occurrence is generated, and compiled into a structured review report.

[0014] Optionally, the timing consistency of the robotic arm's gripping actions is continuously monitored during the sorting process. When the preliminary sorting result is at a critical sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated, including: During the sorting process, the start time, execution trajectory, and completion time of each grasping action of the robotic arm are recorded. Based on the start time, execution trajectory, and completion time, the time interval and trajectory deviation are calculated as timing consistency indicators. Set an upper and lower limit for the sorting threshold. When the similarity metric value corresponding to the preliminary sorting result is between the upper and lower limits, it is determined to be at the sorting threshold threshold. When the timing consistency index exceeds the preset tolerance range, it is determined that the timing consistency of the grabbing action is abnormal. When either the sorting threshold threshold or the timing consistency anomaly occurs, a review instruction is triggered, and the review process is started.

[0015] Secondly, this application provides an automatic sorting and verification system for instant lottery tickets based on super-resolution vision, comprising: The acquisition module is used to acquire clear, continuous video stream images of lottery packages in high-speed motion. The reconstruction module is used to perform super-resolution reconstruction processing on the continuous video stream images. By using dynamic inter-frame alignment and multi-scale feature fusion, it restores the detail information lost on the surface of the lottery package due to motion blur, so as to generate a high-resolution image sequence. The decomposition module is used to extract static visual features of the surface of the lottery package from the high-resolution image sequence, simultaneously acquire the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package, perform spectral decomposition processing on the fluorescence spectral response features to obtain the distribution characteristics of the coating material of the first lottery ticket, combine the static visual features and the distribution characteristics of the coating material to form a multimodal fusion feature, and match and compare the fusion feature with a pre-stored standard feature library to obtain preliminary sorting results; The monitoring module is used to continuously monitor the timing consistency of the robotic arm's gripping actions during the sorting process. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated. The generation module is used to retrospectively capture the fusion features of lottery packages within a specific time period before and after the occurrence of an anomaly. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the module combines the captured anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

[0016] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of an automatic sorting and verification method for instant lottery tickets based on super-resolution vision as described in the first aspect above.

[0017] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the automatic sorting and verification method for instant lottery tickets based on super-resolution vision as described in the first aspect above.

[0018] This application provides an automatic sorting and verification method for instant lottery tickets based on super-resolution vision. By acquiring clear, continuous video stream images of lottery tickets in high-speed motion, it provides fundamental data support for subsequent image processing and feature analysis, avoiding image blurring or discontinuity caused by high-speed motion and ensuring a high-quality source of original images for subsequent processing. Super-resolution reconstruction is performed on the continuous video stream images, using dynamic inter-frame alignment and multi-scale feature fusion to restore details lost due to motion blur, generating a high-resolution image sequence. This compensates for the loss of image details caused by high-speed motion, improving image clarity and resolution, and creating conditions for accurately extracting surface features of the lottery ticket package. Static visual features are extracted from the high-resolution image sequence, and simultaneous multispectral imaging is used to obtain the fluorescence spectral response characteristics of the scratch-off layer coating on the first page of the lottery ticket, which is then spectrally decomposed to obtain the material composition. The system leverages the distribution characteristics of lottery packages, fusing them to form multimodal features, which are then matched with a standard library to obtain preliminary sorting results. This allows for comprehensive capture of lottery package features from both visual and material dimensions, improving the comprehensiveness and accuracy of feature recognition and ensuring the reliability of preliminary sorting results. During sorting, the system continuously monitors the consistency of the robotic arm's grasping timing. When the preliminary sorting results reach a threshold or timing anomalies occur, a review process is initiated. This allows for control over key sorting steps, timely detection of results and action anomalies, prevention of problem escalation, and provides a trigger mechanism for subsequent error correction. By retrospectively analyzing the lottery package features from the abnormal grasping period, if static and dynamic feature judgments are inconsistent, anomaly judgment results are generated by combining the anomaly timestamp and workstation information, and a review report is formed. This accurately locates the root cause of the anomaly and clarifies the anomaly type, providing a basis for optimizing the sorting process and resolving equipment problems, thereby improving the system's fault tolerance and maintainability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating an automatic sorting and verification method for instant lottery tickets based on super-resolution vision, provided for an embodiment of this application; Figure 2 A flowchart illustrating the implementation of an automatic sorting and verification method for instant lottery tickets based on super-resolution vision, provided in this application embodiment; Figure 3 A scene diagram illustrating an automatic sorting and verification method for instant lottery tickets based on super-resolution vision, provided as an embodiment of this application. Figure 4 This is a schematic diagram of the structure of an automatic sorting and verification system for instant lottery tickets based on super-resolution vision, provided in an embodiment of this application. Detailed Implementation

[0021] Current automated sorting systems for instant lottery tickets generally rely on high-speed industrial cameras and traditional image processing algorithms to sort tickets by capturing and matching images of the external markings on the lottery bags. However, this approach has significant limitations: image blurring caused by high-speed movement is difficult to eliminate effectively using traditional filtering algorithms, especially when the surface is worn or reflective, leading to a higher error rate; furthermore, the mechanical control layer and the visual recognition system lack coordination, making it impossible to perceive and dynamically respond to critical sorting states or abnormal grasping actions in real time, heavily relying on manual verification, which restricts the improvement of sorting efficiency and automation level.

[0022] To address the aforementioned issues, this application proposes an automated sorting and verification method for packaged instant lottery tickets based on super-resolution vision and multimodal sensing fusion. This method recovers detailed features from motion-blurred images using super-resolution reconstruction technology and integrates visible light vision and fluorescence spectral features to construct a multimodal recognition model, achieving high-precision non-contact recognition of lottery ticket packages. Furthermore, by monitoring the temporal consistency of the robotic arm's movements and initiating a verification mechanism when the recognition results are critical or abnormal, multi-level verification and anomaly tracing are achieved in the sorting process. This solution significantly improves the sorting accuracy and system robustness of packaged lottery tickets in high-speed environments, reduces the need for manual intervention, and provides a reliable automated sorting solution for operation centers.

[0023] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The core of this application is to provide an automatic sorting and verification method for whole packages of instant lottery tickets based on super-resolution vision. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: S101. Acquire a clear, continuous video stream image of the lottery package under high-speed motion. Optionally, step S101 may specifically include the following steps: S1011. By analyzing the vibration period characteristics of the lottery package during transmission, the relatively stable phase point during the vibration process is determined. S1012. Synchronize the exposure time of the controlled image acquisition device with the relatively stable phase point; S1013. At the synchronous exposure moment, acquire an image of the lottery package in high-speed motion to obtain a clear and continuous video stream image. In the above steps, the lottery package in high-speed motion refers to a whole package of instant lottery tickets moving rapidly on a conveyor device; its position may change during conveyance due to the device's influence. A clear and continuous video stream image is a dynamic image sequence without significant blurring or ghosting, with smooth frame transitions, and fully showcasing the appearance of the lottery package. The vibration periodicity characteristic is the periodic vibration pattern of the lottery package caused by the mechanical operation of the conveyor device, including information such as vibration frequency and amplitude. The relatively stable phase point is the specific moment within the vibration period when the position and attitude of the lottery package change least, and its motion is most stable. The image acquisition device, such as an industrial camera, is used to capture images and has the function of adjusting the exposure time. The exposure time is the specific point in time when the image acquisition device opens the shutter to capture the image.

[0025] In this embodiment, step S1011 first analyzes the vibration period characteristics of the lottery package during transmission to determine a relatively stable phase point. This requires using a vibration sensor to collect vibration data during the transmission of the lottery package, and then using a signal analysis algorithm to process the data, extracting features such as vibration period and amplitude, thereby finding the moment when the position fluctuation of the lottery package within the vibration period is minimal as the relatively stable phase point. For example, if a lottery package of brand A is transmitted by a model B conveyor, the vibration sensor collects that the vibration period of the lottery package driven by the conveyor is 0.2 seconds, and the amplitude varies between 0.5 mm and 2 mm. After processing by the signal analysis algorithm, it is found that at 0.05 seconds and 0.15 seconds of each vibration period, the amplitude of the lottery package is only 0.5 mm, and the position is almost unchanged. These two moments are determined as the relatively stable phase points.

[0026] Secondly, step S1012 synchronizes the exposure time of the image acquisition device with the relatively stable phase point. This process first establishes a signal connection between the image acquisition device and the vibration analysis system, allowing the image acquisition device to receive the time signal of the relatively stable phase point. Then, based on this signal, the device's shutter trigger mechanism is adjusted to ensure the exposure time precisely matches the relatively stable phase point. For example, the 0.05-second and 0.15-second time signals determined in step S1011 are transmitted to a C-type industrial camera. After receiving the signals, the camera adjusts its shutter trigger program, setting it to open the shutter for exposure each time a 0.05-second or 0.15-second signal is detected, thus synchronizing the exposure time with the relatively stable phase point.

[0027] Next, in step S1013, images of the lottery package in high-speed motion are acquired during the synchronous exposure to obtain a clear and continuous video stream. The image acquisition device continuously captures images at the set synchronous exposure times when the lottery package is in a relatively stable state. The time interval between two adjacent acquisitions matches the speed of the lottery package's movement and the required video frame rate, ensuring that the acquired images continuously present the lottery package's movement process without blur. For example, a C-type industrial camera continuously acquires images of a Brand A lottery package at synchronous exposure times of 0.05 seconds, 0.15 seconds, and 0.25 seconds, with a 0.1-second interval between adjacent acquisitions. This matches the speed of the conveyor carrying the lottery package and the 10 frames per second video frame rate. The final acquired images have no obvious blur and continuously present the appearance of the conveyed lottery package, forming a clear and continuous video stream.

[0028] In the overall scheme of step S101 above, by first analyzing the vibration period to determine the stable phase point, then synchronizing the exposure time of the image acquisition device, and finally acquiring the image at the synchronization time, the image blurring problem caused by the high-speed movement and vibration of the lottery bag is effectively avoided, ensuring that the acquired video stream image is clear and continuous. This provides reliable basic data support for subsequent processing such as super-resolution reconstruction and feature extraction of the lottery bag image, and ensures the smooth progress of subsequent steps in the entire automatic lottery sorting and verification process.

[0029] S102. Perform super-resolution reconstruction processing on the continuous video stream images, and restore the detail information lost on the surface of the lottery package due to motion blur by means of dynamic inter-frame alignment and multi-scale feature fusion, so as to generate a high-resolution image sequence. Optionally, step S102 may specifically include the following steps: S1021. Select multiple temporally consecutive frames from the continuous video stream as input frame groups, use the scale-invariant feature transform algorithm to extract feature points of each frame in the input frame group, and use the fast nearest neighbor search matching algorithm to calculate the correspondence between the feature points of different frames. S1022. Based on the feature points and their correspondence, calculate the positional offset between each pair of feature points, and perform geometric transformation on each frame image in the input frame group using an affine transformation algorithm based on the positional offset, so that the relative positions of the lottery package in the multiple frames remain consistent. S1023. Perform a weighted average calculation on the grayscale value of each pixel in the aligned multi-frame image to generate a preliminary fused image. The weights of the weighted average calculation are dynamically adjusted according to the signal-to-noise ratio of each pixel in different frames. S1024. Input the preliminary fused image into a multi-layer processing structure, extract multi-scale features through convolution operation, and obtain feature maps containing different levels of abstraction. S1025. Upsample the feature map to increase its resolution and gradually restore spatial details. S1026. By using skip connections, the low-level detailed features are fused with the upsampled high-level semantic features to generate an enhanced feature representation; S1027. Based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by dynamic trailing are reconstructed layer by layer, the text outline of the category identification and the detailed features of the micro anti-counterfeiting pattern are restored, and a high-resolution image sequence is output.

[0030] Step S1027 may specifically include the following processes: separating the basic features of the edge texture and the detailed features of the high-frequency information from the enhanced feature representation; based on the basic features of the edge texture, supplementing the missing edge segments and connecting broken contour lines on the surface of the lottery package in the dynamic shadow coverage area, and filling the detailed features of the high-frequency information within the edge texture framework to restore the texture's light and dark transitions and subtle undulations; based on the text-related feature components in the enhanced feature representation, strengthening the contrast of the text stroke edges and repairing broken parts in the category identification area to form a continuous closed text outline; based on feature points with a size smaller than a preset threshold in the enhanced feature representation, connecting adjacent feature points in the miniature anti-counterfeiting pattern area according to the pattern arrangement rules to form a complete pattern unit and supplementing the connection structure between units; integrating the restored edge texture, high-frequency information, category identification text outline, and miniature anti-counterfeiting pattern into a single-frame high-resolution image, and arranging all the generated single-frame high-resolution images in chronological order to form a high-resolution image sequence.

[0031] In the above steps, continuous video stream images refer to the sequence of dynamic lottery package images acquired in step S101 that are free of obvious blur and have smooth inter-frame transitions; super-resolution reconstruction processing is the process of improving image resolution through algorithms and restoring detail information lost due to various factors; dynamic inter-frame alignment refers to adjusting the relative positions of lottery packages in each frame of the continuous video stream image to keep them consistent; multi-scale feature fusion extracts and integrates image features from different resolutions and different levels of abstraction to comprehensively present image information; scale-invariant feature transformation algorithm is used to extract feature points in the image that are not affected by factors such as scale, rotation, and illumination; fast nearest neighbor search matching algorithm can quickly find the correspondence between feature points in different frames of images; affine transformation algorithm can perform operations such as translation, rotation, scaling, and cropping on the image while maintaining the affine nature of the image; grayscale value refers to the brightness information of each pixel in the image; information Noise ratio measures the proportion of effective signal to noise in a signal; a multi-layer processing structure is a structure composed of multiple processing layers, usually used for image feature extraction; convolution operation is a weighted summation operation performed by sliding a convolution kernel across the image, which can extract image features; a feature map is a matrix containing different features of the image, obtained after processing such as convolution operation; upsampling operation increases image resolution and restores spatial details of the image through specific algorithms; skip connections are used to connect feature maps at different levels and fuse features at different levels; enhanced feature representation is a feature description containing richer and more accurate image information after processing such as feature fusion; edge texture refers to the shape, texture, and other features of the edges of objects in an image; high-frequency information contains information about details, edges, and other drastically changing features in the image; category identifiers are used to indicate the type of lottery package; micro anti-counterfeiting patterns are tiny patterns used on lottery packages for anti-counterfeiting purposes.

[0032] In this embodiment, firstly, step S1021 selects multiple temporally consecutive frames from the continuous video stream as input frame groups. A scale-invariant feature transform algorithm is used to extract feature points from each frame in the input frame group, and a fast nearest neighbor search matching algorithm is used to calculate the correspondence between feature points in different frames. For example, five adjacent frames are selected from the continuous video stream of a lottery bag as input frame groups. Using the scale-invariant feature transform algorithm, stable feature points such as the corners of the lottery bag and the edges of text are found in each frame. For instance, if a feature point is detected at the corner of a text on the lottery bag in one frame, a corresponding feature point can be detected at the same corner in another frame. Then, the fast nearest neighbor search matching algorithm is used to quickly determine the correspondence between these feature points in different frames, such as determining the corresponding feature point in the second frame for a feature point in the first frame.

[0033] Secondly, in step S1022, the positional offset between each pair of feature points is calculated based on the feature points and their correspondences. Then, an affine transformation algorithm is used to perform a geometric transformation on each frame in the input frame group based on these offsets, ensuring that the relative positions of the lottery packages in the multiple frames remain consistent. For example, assuming that step S1021 determines that feature point A in the first frame corresponds to feature point A' in the second frame, a horizontal offset of 5 pixels and a vertical offset of 3 pixels are calculated. Using these offsets, an affine transformation algorithm is applied to the second frame image to perform translation, rotation, or scaling operations, making the lottery packages in the second frame image as close as possible to the lottery packages in the first frame image in relative position. If the lottery package is upright in the first frame image, the affine transformation makes the slightly tilted lottery package in the second frame image also upright, consistent with the position and orientation of the lottery package in the first frame.

[0034] Next, in step S1023, the grayscale values ​​of each pixel in the aligned multi-frame images are calculated using a weighted average to generate a preliminary fused image. The weights for the weighted average calculation are dynamically adjusted based on the signal-to-noise ratio (SNR) of each pixel in different frames. For example, for a pixel at a certain position in the aligned 5-frame images, if the SNR of that pixel is high in the first frame, it indicates that it is less affected by noise and contains more effective information. Therefore, when calculating the weighted average, the grayscale value of that pixel in the first frame is given a higher weight. If the SNR of that pixel is low in another frame, it is more affected by noise and is given a lower weight. By combining the grayscale values ​​of that pixel in each frame with their corresponding weights, a weighted average is calculated and used as the grayscale value of that pixel in the preliminary fused image, thus generating the final preliminary fused image.

[0035] Next, in step S1024, the initially fused image is input into a multi-layer processing structure. Multi-scale features are extracted through convolution operations to obtain feature maps containing different levels of abstraction. For example, the initially fused image is input into a multi-layer processing structure composed of multiple convolutional layers. The first convolutional layer uses a small-sized convolutional kernel to extract detailed features such as edges and textures in the image, obtaining feature maps that reflect these details. Subsequent convolutional layers use larger-sized convolutional kernels to gradually extract more abstract features such as object shape and overall layout, obtaining feature maps containing features at different levels. For example, one feature map mainly highlights the overall shape of the lottery bag, while another feature map highlights the texture details on the surface of the lottery bag.

[0036] Then, in step S1025, the feature map is upsampled to increase its resolution, gradually restoring spatial details. For example, for a feature map containing the overall shape features of a lottery bag, an upsampling operation, such as bilinear interpolation, is used to increase the number of pixels in the feature map without changing the feature information. This makes the originally blurry edges of the lottery bag shape clearer, and some small details that were originally missing gradually appear, such as the fine lines on the lottery bag becoming more coherent.

[0037] Next, in step S1026, low-level detail features are fused with upsampled high-level semantic features via skip connections to generate an enhanced feature representation. For example, the low-level edge texture and other detail feature maps extracted by the first convolutional layer in step S1024 are skip-connected with the upsampled high-level feature maps, such as those highlighting the overall shape of the lottery bag and the approximate location of the category identifier. This integrates the low-level detail features into the high-level semantic features, forming an enhanced feature representation that contains more comprehensive information. This allows the details and overall features of the lottery bag to be better combined, such as making the positional relationship between the micro-anti-counterfeiting pattern details on the lottery bag and the overall category identifier more clearly apparent.

[0038] Finally, through step S1027, based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by the dynamic trailing shadow are reconstructed layer by layer, restoring the text outline of the category identifier and the detailed features of the miniature anti-counterfeiting pattern, and outputting a high-resolution image sequence. Specifically, the basic features of the edge texture and the detailed features of the high-frequency information are separated from the enhanced feature representation; based on the basic features of the edge texture, in the area covered by the dynamic trailing shadow, missing edge segments and connecting broken contour lines are added to the surface of the lottery package, and the detailed features of the high-frequency information are filled within the edge texture framework to restore the texture's light and dark transitions and subtle undulations; based on the text-related feature components in the enhanced feature representation, the contrast of the text stroke edges is strengthened and broken parts are repaired in the category identifier area to form a continuous closed text outline; based on feature points with a size smaller than a preset threshold in the enhanced feature representation, adjacent feature points are connected in the miniature anti-counterfeiting pattern area according to the pattern arrangement rule to form complete pattern units and supplement the connection structure between units; the restored edge texture, high-frequency information, category identifier text outline, and miniature anti-counterfeiting pattern are integrated into a single high-resolution image, and all generated single high-resolution images are arranged in chronological order to form a high-resolution image sequence. For example, in the enhanced feature representation, a missing line segment due to dynamic blurring is detected at the edge of the lottery package. Based on the basic features of the edge texture, this line segment is supplemented, the broken contour lines are connected, and high-frequency information is filled in to make the light and dark transitions of the edge texture more natural. For category identification text, if any strokes are blurred or broken, the text-related feature components are used to enhance the contrast of the stroke edges, repair the broken parts, and form a complete text outline. In the micro-anti-counterfeiting pattern area, feature points smaller than a preset threshold are found, and adjacent feature points are connected according to the pattern arrangement to form complete pattern units and supplement the connection structure between units. These restored parts are integrated into a single high-resolution image, and each frame is processed sequentially and arranged in chronological order to form a high-resolution image sequence that clearly displays the details of the lottery package.

[0039] In a practical application scenario, when a lottery processing agency performs image analysis on a batch of lottery packages, it first acquires a continuous video stream of the packages moving rapidly on a conveyor belt. Ten consecutive frames are selected as input frames. Due to conveyor belt vibration and the high-speed movement of the lottery packages, these images exhibit differences in position and pose, as well as motion blur. The agency first uses a scale-invariant feature transform algorithm to detect feature points such as the corners of text and the turning points of patterns on the lottery packages in each frame. For example, if a corner feature point of the word "lucky" is identified in one frame, the corresponding feature point is found in another frame. Then, a fast nearest neighbor search matching algorithm is used to calculate the similarity of feature vectors, determining the correspondence between feature points in different frames. Next, based on these correspondences, the horizontal and vertical offsets of feature point A in the first frame and feature point A' in the second frame are calculated. An affine transformation algorithm is used to construct a matrix, and translation and rotation operations are performed on each frame to ensure consistent position and pose of the lottery packages. Finally, the weights of each pixel in the aligned 10 frames are dynamically adjusted according to the signal-to-noise ratio (SNR) of each pixel in different frames. For example, a pixel with an SNR of 80 in the third frame is assigned a weight of 0.6, and so on. Five frames with a signal-to-noise ratio of 50 are assigned a weight of 0.2 and weighted averaged to generate a preliminary fused image. This image is then input into a 5-layer convolutional structure. The first layer uses a 3×3 convolutional kernel to extract edge texture features, while subsequent layers use 5×5 and 7×7 convolutional kernels to extract abstract features such as overall shape. The feature map is upsampled using bilinear interpolation to improve resolution. Skip connections are used to fuse low-level details and high-level semantic features to form an enhanced feature representation. Finally, edge texture and high-frequency information features in the enhanced features are separated. Missing line segments on the edge of the lottery package are repaired, texture details are filled in, the edges of the category identification text are strengthened, and broken parts are repaired. Micro-anti-counterfeiting pattern feature points are connected according to the arrangement rules to form complete units. These are integrated into a single high-resolution image and arranged into a sequence in chronological order to provide clear data for subsequent processing.

[0040] In the overall scheme of step S102 above, through the collaborative processing of multiple steps and algorithms, from feature point extraction and matching, to image alignment and fusion, then to feature extraction, resolution enhancement and feature fusion, and finally the reconstruction of damaged details, the detailed information lost by motion blur on the surface of the lottery bag is effectively restored, and a high-resolution image sequence is generated. This provides a high-quality image foundation for subsequent processes such as accurate extraction of static visual features of the lottery bag, multispectral imaging analysis and sorting verification, and improves the accuracy and reliability of the entire automatic lottery sorting verification system for lottery bag detail recognition.

[0041] S103. Extract the static visual features of the lottery package surface from the high-resolution image sequence, simultaneously obtain the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery package's homepage lottery ticket, perform spectral decomposition processing on the fluorescence spectral response features to obtain the distribution characteristics of the homepage lottery ticket coating material, combine the static visual features with the distribution characteristics of the coating material to form a multimodal fusion feature, match and compare the fusion feature with a pre-stored standard feature library to obtain preliminary sorting results; As an alternative, such as Figure 2 As shown, the process of "obtaining the fluorescence spectral response characteristics of the scratch-off layer coating corresponding to the lottery ticket on the homepage, performing spectral decomposition processing on the fluorescence spectral response characteristics to obtain the distribution characteristics of the coating material of the homepage lottery ticket, and combining the static visual characteristics with the distribution characteristics of the coating material to form a multimodal fusion feature" in step S103 can specifically include the following steps: S1031. Based on the positioning information of the high-resolution image sequence, control the multispectral optical imaging system to excite the coating layer of the lottery ticket using multiple light sources of different wavelengths in a time-division polling manner. S1032. The multispectral optical imaging system is used to acquire fluorescence images of the excited scraped layer in multiple spectral bands, and a continuous spectral reflectance curve for each pixel is generated based on the fluorescence images. S1033. Decompose the spectral reflectance curve into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum, and calculate the contribution of each spectral component to the pixel. Specifically, step S1033 may include the following process: establishing a corresponding spectral reflectance curve for each pixel of the scratch-off layer of the lottery package homepage, wherein the spectral reflectance curve is composed of the reflectance intensity values ​​of multiple spectral bands, and is represented as a weighted sum of the corresponding components of the standard spectral of the coating, the corresponding components of the paper substrate spectral, and the corresponding components of the printing ink spectral; solving for the weight coefficients of the corresponding components of the standard spectral of the coating, the corresponding components of the paper substrate spectral, and the corresponding components of the printing ink spectral by minimizing the difference between the spectral reflectance curve and the weighted sum; and using the weight coefficients as the contribution of each spectral component to the corresponding pixel.

[0042] S1034. Based on the contribution analysis, the continuous variation characteristics of the standard spectrum of the coating in spatial distribution are analyzed to identify whether there are uneven thickness or abnormal coverage areas in the scratch-off layer of the lottery ticket on the homepage as the coating anomaly identification result. At the same time, based on the temporal variation of multiple frames in the high-resolution image sequence, the motion posture change characteristics of the lottery ticket package are analyzed. Specifically, step S1034 may include the following processes: arranging the contribution of the corresponding components of the coating standard spectrum in each pixel according to their actual positions; detecting the contribution changes of adjacent pixels and recording a continuous numerical sequence; determining the spatial continuity variation characteristics of the coating standard spectrum through the fluctuation amplitude and change frequency of the continuous numerical sequence; based on the continuous variation characteristics, identifying areas where the contribution variation amplitude of multiple consecutive pixels exceeds a preset range, and where the overall contribution of the area differs significantly from the surrounding area, as areas with uneven thickness of the scratch-off layer; identifying local areas where the contribution value is zero or far exceeds the normal range, and where the contribution value differs significantly from the surrounding area. The region with abrupt boundary is designated as an abnormal coverage area of ​​the scratch-off layer. The location information of the uneven thickness area and the abnormal coverage area are integrated to form an abnormal coating identification result. Fixed marker points on the surface of the lottery bag are selected from the high-resolution image sequence, and the coordinate information of the fixed marker points on the surface of the lottery bag is obtained by tracking the position changes of the marker points in each frame image. Based on the coordinate information, the relative change of the marker point coordinates in adjacent frames is calculated, and the rotation angle and displacement distance of the lottery bag per unit time are determined based on the relative change. According to the change law of the rotation angle and displacement distance of the lottery bag, the motion posture change characteristics of the lottery bag are formed.

[0043] S1035. The static visual features, the film overlay anomaly recognition results, and the motion posture change features are fused in a multimodal manner to form fused features.

[0044] In step S103, the high-resolution image sequence is an image sequence generated in step S102 that clearly displays the details of the lottery package; static visual features refer to the features that can be directly observed on the surface of the lottery package, such as shape, color, and pattern; multispectral imaging obtains information about the object in different spectral bands by exciting it with multiple light sources of different wavelengths; fluorescence spectral response features are the spectral characteristics of the fluorescence generated after the scratch-off coating of the lottery package is excited; spectral decomposition processing separates the mixed spectrum into different component spectra; the distribution characteristics of the coating material describe the spatial distribution of the coating material of the lottery package; multimodal fusion features are comprehensive features obtained by fusing different types of features; the pre-stored standard feature library is a set of various standard features stored in advance for comparison; positioning information is used to determine the position of the lottery package in the image; and the multispectral optical imaging system is used for... The equipment is designed for multispectral imaging; it uses a time-sharing polling method to sequentially switch between different wavelength light sources; the continuous spectral reflectance curve reflects the continuous change in reflectance intensity of each pixel across multiple spectral bands; the standard spectrum of the coating material is the spectral characteristic of the coating material under ideal conditions; the paper substrate spectrum is the spectral characteristic of the lottery package paper itself; the printing ink spectrum is the spectral characteristic of the printing ink on the lottery package; the contribution degree indicates the relative importance of each spectral component in the pixel's spectral reflectance curve; the distribution characteristics of the coating material refer to the continuous change characteristics of the standard spectral reflectance in spatial distribution based on the contribution degree analysis; the weighting coefficient is used to measure the proportion of each spectral component in the weighted sum; the continuous change characteristics in spatial distribution describe the variation law of the standard spectral reflectance in spatial location; and the motion posture change characteristics reflect the posture change of the lottery package during movement.

[0045] In this embodiment, firstly, step S1031 determines the position of the lottery package in the image based on the positioning information of the high-resolution image sequence. The multispectral optical imaging system is then controlled to operate in a time-division polling manner. This system sequentially uses multiple light sources of different wavelengths, such as 400nm, 500nm, and 600nm, to excite the scratch-off layer coating on the front of the lottery package according to a preset sequence. For example, when the positioning information indicates that the lottery package is slightly to the left of the center of the image, the multispectral optical imaging system will precisely illuminate the scratch-off layer coating at that location with different wavelengths of light in sequence.

[0046] Secondly, in step S1032, a multispectral optical imaging system is used to acquire fluorescence images of the excited scratch-off layer in multiple spectral bands. This system can acquire image information of the scratch-off layer under different light source excitation. Then, based on these fluorescence images, for each pixel, a continuous spectral reflectance curve is generated by analyzing its reflectance intensity in each spectral band. For example, for a certain pixel on the scratch-off layer coating in the image, the reflectance intensity is 50 under a 400nm wavelength light source excitation, 60 under a 500nm wavelength, and 70 under a 600nm wavelength, etc. Connecting these intensity values ​​in spectral band order yields the continuous spectral reflectance curve of that pixel.

[0047] Next, in step S1033, the spectral reflectance curve is decomposed into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum. For each pixel of the scratch-off layer coating on the lottery ticket's homepage, a corresponding spectral reflectance curve model is established, which is represented as: ,in, Represents the pixel at wavelength The spectral reflectance intensity values ​​at a given location constitute the continuous spectral reflectance curve for that pixel. These are the weighting coefficients for the standard spectrum of the coating. It is the weighting coefficient of the paper substrate spectrum. These are the weighting coefficients of the printing ink spectrum, and together they reflect the contribution ratio of each spectral component in the pixel spectrum. Indicates at wavelength The standard spectral intensity of the coating at the location, that is, the spectral characteristics of the coating material when it exists alone under ideal conditions; Indicates at wavelength The spectral intensity of the paper substrate at that location, i.e., the spectral characteristics of the lottery package paper itself; Indicates at wavelength The spectral intensity of the printed ink at a certain location refers to the spectral characteristics of the printed ink on the surface of the lottery ticket package. The least squares algorithm is used to calculate the actual measured... Minimize the sum of squared errors between the weighted sum on the right side of the formula and the sum of squared errors. The weighting coefficients obtained by solving are the contribution of each spectral component to the corresponding pixel. For example, after calculation, for pixel... This indicates that in the spectral reflectance curve of this pixel, the contribution of the standard spectrum of the coating is 0.4, the contribution of the paper substrate spectrum is 0.3, and the contribution of the printing ink spectrum is 0.3. Based on this, the material composition of this point can be determined.

[0048] Then, in step S1034, the continuity variation characteristics of the coating standard spectrum in spatial distribution are analyzed based on contribution. The values ​​of each pixel are then... The standard spectral weighting coefficients for the coating, i.e., the contribution, are arranged sequentially according to the actual position coordinates of pixels in the image, such as the x-axis horizontally and the y-axis vertically, forming a two-dimensional contribution matrix. A sliding window algorithm is used to detect adjacent pixels, such as the four adjacent pixels (top, bottom, left, and right). The difference is used to record the changes of each pixel relative to its surrounding pixels, forming a continuous numerical sequence. The fluctuation range of this sequence is analyzed, such as that between adjacent pixels. The maximum value and frequency of change of the difference, such as the number of times the difference exceeds a preset threshold within a unit length, determine the spatial continuity of the standard spectrum of the coating. If there are more than 10 consecutive pixels... If the change exceeds the preset range, such as a difference greater than 0.1, and the entire area... If the difference between the average value and the average value of the surrounding area exceeds 0.2, it is determined to be an area with uneven scraping layer thickness; if the difference is local area pixel range A value of 0 or greater than 0.8 is found, far exceeding the normal range of 0.2- Furthermore, it forms a clear numerical abrupt boundary with surrounding pixels, and adjacent pixels If the difference is greater than 0.5, it is determined to be an abnormal area of ​​the scratch-off layer coverage. The coordinate ranges of these areas are integrated to form the coating anomaly identification result. Simultaneously, fixed marker points on the surface of the lottery bag are selected from the high-resolution image sequence, such as the vertex M of the triangle pattern in the corner of the lottery bag. The position of M in each frame image is tracked using a feature matching algorithm, and the coordinates of M in each frame image are recorded. (n is the frame number). Calculate the relationship between the nth frame and the frame number. The relative change in the coordinates of M in the frame ,based on and The rotation angle of the lottery package is calculated using trigonometric functions. The displacement distance is calculated using the Euclidean distance formula. Based on multiple consecutive frames and Patterns of change, such as The continuous increase in size indicates that the lottery package is rotating clockwise. Stability and invariance indicate uniform displacement velocity, forming the characteristic of the lottery package's motion posture changes.

[0049] Finally, in step S1035, the static visual features, the results of the coating anomaly identification, and the motion posture change features are fused in a multimodal manner. The static visual features such as the shape and color of the lottery package, the results of the coating anomaly identification such as uneven thickness and abnormal coverage of the scratch-off layer on the first page of the lottery ticket, and the motion posture change features such as the rotation angle and displacement distance during the movement of the lottery package are integrated together using a specific fusion algorithm, such as a weighted fusion algorithm, to form a fused feature.

[0050] In a practical application, a lottery processing company performs pre-sorting inspection on packages of Type A instant lottery tickets. First, the positioning coordinates of the lottery packages on the conveyor belt are determined based on high-resolution image sequences, such as in the image coordinate system. to Within a rectangular region, the multispectral optical imaging system is controlled to sequentially activate in a time-division polling manner. An LED light source of a specific wavelength is used to excite the scratch-off coating of the lottery ticket within the rectangular area. The system simultaneously acquires fluorescence images at each wavelength, extracts the reflection intensity values ​​corresponding to the four wavelengths for each pixel in the image, and generates a continuous spectral reflectance curve. By calculating the weighting coefficients of each pixel using the spectral decomposition formula, a patch was found on the left edge of the lottery ticket package. The area of ​​pixels, The range of change is Between, and regional average The average surrounding area It was determined to be an area of ​​uneven thickness; there was one in the center of the lottery bag. The area of ​​pixels, Surrounding This area was identified as having abnormal coverage. Simultaneously, the circular marker in the upper right corner of the lottery package was selected, and its coordinates were tracked across 10 frames of images. The average rotation angle between adjacent frames was calculated. The average displacement distance was 5mm, confirming stable motion. Finally, the dimensions of the lottery bag were determined. The surface red main color, abnormal area information and motion posture data are fused to form a multimodal fusion feature, which is used for subsequent matching with the standard feature library.

[0051] In the overall scheme of step S103 above, multispectral imaging and spectral decomposition technology are used to accurately obtain the material distribution characteristics of the scratch-off layer coating of the lottery ticket, effectively identifying abnormal coating areas; static visual features are extracted by combining high-resolution images, while tracking the movement and posture changes of the marker points; finally, multimodal fusion is used to integrate multi-dimensional information to form comprehensive and accurate fused features. This scheme provides high-quality data support for subsequent matching and comparison with the pre-stored standard feature library, avoids the limitations of single feature analysis, significantly improves the accuracy and reliability of the initial sorting results, and lays a key foundation for the automatic lottery ticket sorting and verification process.

[0052] As another optional solution, after step S102, the method further includes: obtaining the optical characteristic parameters of the transparent heat-shrinkable film of the lottery package outer packaging, including the transmittance distribution characteristics and surface refractive characteristics; Therefore, in another alternative approach, step S103 can also extract static visual features of the lottery package surface from the high-resolution image sequence, and simultaneously acquire the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package; combine the static visual features, the distribution characteristics of the coating material of the first lottery ticket, and the optical property parameters of the transparent heat-shrink film to form a multimodal fusion feature; match and compare the multimodal fusion feature with the pre-stored standard feature library, and comprehensively judge the quality of the entire package of lottery tickets based on the coating status of the first lottery ticket and the integrity of the outer packaging to obtain the preliminary sorting result.

[0053] Similar to the above options, the difference lies in introducing optical property parameters of the outer packaging of the lottery package (i.e., transparent heat-shrink film) to further improve the accuracy of sorting results.

[0054] In the above steps, optical characteristic parameters are key data describing the optical performance of the transparent heat-shrink film on the outer packaging of lottery tickets, including transmittance distribution characteristics and surface refractive properties. Transmittance distribution characteristics refer to the difference in the proportion of light allowed to pass through different areas of the transparent heat-shrink film, reflecting the impact of uneven film thickness or stains on light. Surface refractive properties refer to the variation of the refraction angle when light passes through the surface of the heat-shrink film, which is related to the smoothness of the film surface. The distribution characteristics of the material of the first lottery ticket coating refer to the material type of the outermost first lottery ticket covering the entire package, such as PET, PVC, coverage area, and thickness distribution. Multimodal fusion features are comprehensive features that integrate static visual features, including lottery ticket color, pattern, etc., the material distribution characteristics of the first lottery ticket coating, and the optical characteristic parameters of the transparent heat-shrink film, forming a comprehensive feature containing visual, material, and optical multi-dimensional information. The condition of the first lottery ticket coating refers to the integrity of the first lottery ticket coating, such as whether there is damage or wrinkles; the integrity of the outer packaging refers to whether the transparent heat-shrink film completely wraps the lottery ticket package, and whether there are tears, openings, etc.

[0055] In this embodiment, after performing super-resolution reconstruction on continuous video stream images and generating a high-resolution image sequence by restoring surface details of the lottery package through dynamic inter-frame alignment and multi-scale feature fusion, the optical information of the transparent heat-shrink film is first extracted from the high-resolution image sequence. This is combined with an optical detection module to obtain the transmittance distribution characteristics determined by analyzing differences in light transmission intensity in different regions, and the surface refractive properties obtained by calculating changes in the light refraction angle, thus forming the optical characteristic parameters of the transparent heat-shrink film for the outer packaging of the lottery package. Next, the first lottery ticket on the outermost layer of the entire package is located, and its coating material type is determined using material recognition algorithms such as spectral analysis-assisted image recognition. Simultaneously, the coverage area and thickness of the coating are analyzed. The distribution characteristics of the first lottery ticket's coating material are obtained. Then, the previously extracted static visual features of the lottery package are retrieved. The static visual features, the distribution characteristics of the first lottery ticket's coating material, and the optical characteristics of the transparent heat-shrink film are combined according to preset weights, such as static features accounting for 0.4, coating distribution characteristics accounting for 0.3, and optical parameters accounting for 0.3, to construct a multimodal fusion feature. Finally, the multimodal fusion feature is matched and compared with a pre-stored standard feature library, which contains multimodal standard templates for lottery tickets of different categories and quality states. Based on the matching results, it is determined whether the coating of the first lottery ticket is intact and whether the outer transparent heat-shrink film is intact. The overall quality of the lottery package is comprehensively evaluated, and a preliminary sorting result including quality judgment is finally obtained.

[0056] In practical applications, after generating a 5120×2880 high-resolution image sequence through super-resolution processing, the lottery sorting line of Brand A initiates a multimodal feature processing workflow. The optical detection module configured on this sorting line first analyzes the transparent heat-shrink film of the lottery packages in the high-resolution images, detecting that the light transmittance of the top area of ​​the film is 85% and the light transmittance of the side area is 90%, i.e., the light transmittance distribution characteristics. Simultaneously, it calculates the average refraction angle deviation of 1.2° when light passes through the film, i.e., the surface refraction characteristics, forming optical characteristic parameters. Next, the system locates the first lottery ticket in the outermost layer of the entire package. Through spectral analysis combined with image texture recognition, it determines that the film material is PET, and that the film completely covers 100% of the first lottery ticket's coverage area with an average thickness of 0.02mm, which is taken as the distribution characteristics of the first lottery ticket's film material. Subsequently, it retrieves the previously extracted static visual features—red and yellow background, circular five-pointed star anti-counterfeiting pattern—and combines them into multimodal fusion features with weights of 0.4 for static features, 0.3 for film distribution characteristics, and 0.3 for optical parameters. The feature was matched with the standard feature library and found to have a similarity of 0.92 with the standard template "Lucky Lottery - 10 Yuan". The matching results showed that the first lottery ticket had no damaged film, no torn transparent heat shrink film, and intact outer packaging. Based on the overall judgment, the quality of the whole package of lottery tickets was qualified, and the preliminary sorting result was "Category: Lucky Lottery - 10 Yuan, Quality: Qualified".

[0057] In the overall solution described above, by integrating the optical properties of the newly added transparent heat-shrink film with the material distribution characteristics of the first lottery ticket's lamination, the limitations of relying solely on visual features are overcome, enabling a multi-dimensional assessment of the lottery ticket package's "appearance + packaging quality." This not only accurately identifies lottery ticket categories but also promptly detects quality issues such as damaged lamination on the first lottery ticket and torn outer heat-shrink film, preventing substandard tickets from entering subsequent stages. The construction of multi-modal fusion features enhances the feature dimensions and reduces interference from stains and slight blurring of individual features. This ensures that the initial sorting results not only contain category information but also possess quality judgment value, further improving the completeness and reliability of the sorting process and meeting the lottery industry's stringent product quality control requirements.

[0058] S104. During the sorting process, continuously monitor the timing consistency of the robotic arm's gripping actions. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, initiate the review procedure. Optionally, step S104 may specifically include the following steps: S1041. During the sorting process, record the start time, execution trajectory, and completion time of each grasping action of the robotic arm. Based on the start time, execution trajectory, and completion time, calculate the time interval and trajectory deviation as timing consistency indicators. S1042. Set an upper and lower limit for the sorting threshold. When the similarity metric value corresponding to the preliminary sorting result is between the upper and lower limits, it is determined to be the sorting threshold threshold critical. S1043. When the timing consistency index exceeds the preset tolerance range, it is determined that the timing consistency of the grasping action is abnormal. S1044. When either the sorting threshold threshold or the timing consistency anomaly occurs, a review instruction is triggered to start the review process.

[0059] In the above steps, the start time of the robotic arm's grasping action is the moment when the robotic arm begins to perform the grasping action; the execution trajectory is the path the robotic arm moves along during the grasping process; the completion time refers to the moment the robotic arm completes the grasping action. The time interval is the difference in duration between the start time and the completion time, used to measure the time span of the grasping action; the trajectory deviation indicates the degree of deviation between the actual execution trajectory of the robotic arm and the preset standard trajectory. These two together constitute the timing consistency index. The upper and lower limits of the sorting threshold are pre-set numerical boundaries used to determine whether the preliminary sorting result is in a critical state; the similarity metric is a quantitative representation of the similarity between the preliminary sorting result and the standard sorting result; when this value is between the upper and lower limits, it is determined that the sorting threshold is critical. The preset tolerance range is the allowable fluctuation range set for the timing consistency index. When the index exceeds this range, it is determined that the timing consistency of the grasping action is abnormal. The review instruction is the command signal that triggers the start of the review program.

[0060] In this embodiment, firstly, in step S1041, during the sorting process, a high-precision timing device is used to record the starting time of each grasping action of the robotic arm. For example, in a certain grasping action, the starting time is T1. Simultaneously, using a motion trajectory tracking system, such as tracking technology based on visual recognition or sensor feedback, the execution trajectory of the robotic arm in three-dimensional space is acquired. Assume its trajectory is a curve from coordinate point A(x1,y1,z1) to coordinate point B(x2,y2,z2). When the robotic arm completes the grasping action, the completion time is recorded again using the timing device, denoted as T2. Based on the recorded starting time T1 and completion time T2, the time interval is calculated as ΔT = T2 - T1. For trajectory deviation, the actual acquired execution trajectory is compared with a pre-stored standard trajectory. A specific trajectory matching algorithm, such as an algorithm based on keypoint matching or curve fitting, is used to calculate the degree of deviation between the actual trajectory and the standard trajectory in various dimensions, and a comprehensive trajectory deviation value is obtained.

[0061] Secondly, in step S1042, based on the requirements and experience of the actual sorting task, an upper limit of the sorting threshold is set as U, and a lower limit as L. After obtaining the preliminary sorting results, a similarity calculation algorithm, such as the cosine similarity algorithm or the Euclidean distance algorithm, is used to compare the preliminary sorting results with the standard sorting results to obtain a similarity metric S. When S satisfies L≤S≤U, it is determined to be at the critical sorting threshold. For example, if the calculated similarity metric S is 0.6, and the set lower limit L is 0.5 and the upper limit U is 0.7, then the condition is met, and it is determined to be at the critical sorting threshold.

[0062] Next, in step S1043, the timing consistency index, composed of the time interval and trajectory deviation calculated in step S1041, is compared with a preset tolerance range. Assume the preset tolerance range for the time interval is [ΔTmin, ΔTmax], and the tolerance range for the trajectory deviation is [Dmin, Dmax]. When the time interval ΔT is not within [ΔTmin, ΔTmax], or the trajectory deviation value is not within [Dmin, Dmax], it is determined that the timing consistency of the grasping action is abnormal. For example, if the calculated time interval ΔT is 3 seconds, while the preset tolerance range is [2, 2.5] seconds, the time interval exceeds the tolerance range, and it is determined to be a timing consistency abnormality.

[0063] Finally, through step S1044, when either a sorting threshold threshold or a timing consistency anomaly occurs, the system automatically generates a review instruction. This instruction is sent to the relevant control module via the communication interface to initiate the review process. For example, if a timing consistency anomaly is detected at a certain moment, the system generates a review instruction. Upon receiving the instruction, the control module begins allocating resources to prepare for initiating the review process.

[0064] In a practical application scenario, a lottery processing company undertook the sorting task of type A instant lottery tickets in full packages. First, at the beginning of the sorting line, based on the high-resolution image sequence generated in step S102, an image recognition algorithm was used to accurately determine the positioning coordinates of the lottery packages on the conveyor belt, obtaining the coordinates from the image coordinate system. to The rectangular area is the location of the lottery package. Subsequently, the multispectral optical imaging system is operated in a time-sharing polling manner, activating sequentially according to a predetermined order. An LED light source of varying wavelengths is used to excite the scratch-off layer coating of the lottery package within the rectangular area one by one. The image sensor in the system quickly captures the fluorescence signals reflected from the scratch-off layer under different light source excitations and converts them into digital image information. For each pixel in each fluorescence image, the reflection intensity values ​​corresponding to the four wavelengths are carefully extracted. For example, at a certain pixel, the reflection intensity is 48 (grayscale value) under 420nm light source excitation, and is [missing value] under 520nm light source excitation. The value below 63 nm and below 720 nm is 58 nm. These intensity values ​​are concatenated in ascending order of wavelength to successfully generate a continuous spectral reflectance curve. Using spectral decomposition formulas and performing complex calculations on each pixel, a patch was discovered on the left edge of the lottery package. The region of pixels, within this region The variation range of the standard spectral weighting coefficient for film coating is between 0.12 and 0.25, and the regional average is... The average in the surrounding area This indicates an area of ​​uneven thickness; there is one in the center of the lottery package. The area of ​​pixels, here Surrounding This led to the determination that the area was an abnormally covered region. Simultaneously, the circular marker in the upper right corner of the lottery package was selected, and its coordinate changes across 10 frames were closely tracked using a feature matching algorithm. The calculated average rotation angle between adjacent frames was... The average displacement distance was 5mm, confirming that the lottery bag's motion was relatively stable. Finally, the dimensions of the lottery bag were measured as follows: Static visual features such as the dominant red color on the surface are fused with information on identified abnormal areas and motion posture data to carefully construct multimodal fusion features, providing comprehensive and accurate data support for subsequent matching and comparison with standard feature libraries.

[0065] In the overall scheme of step S104 above, by monitoring the key parameters of the robotic arm's gripping action, abnormal situations during the gripping process and critical states of the preliminary sorting results can be detected in a timely manner. When an abnormality or critical situation occurs, a review procedure is quickly initiated to prevent problematic sorting results from entering the next stage, thereby improving the reliability and accuracy of the sorting process and ensuring the efficient and stable operation of the entire sorting process.

[0066] S105. Retrospectively capture the fusion features of lottery packages within a specific time period before and after the occurrence of the anomaly. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, combine the capture anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

[0067] As an optional approach (i.e., obtaining the judgment result of the static visual features and the judgment result based on the distribution characteristics of the coating material), step S105 may specifically include the following steps: S1051. Backtrack and capture the fusion features of the lottery package within a specific time period before and after the anomaly occurred, and analyze them based on the static visual features and the distribution characteristics of the coating material in the fusion features, respectively, to obtain the judgment result of the static visual features and the judgment result based on the distribution characteristics of the coating material. S1052. Compare the judgment result of the static visual feature with the judgment result based on the distribution characteristics of the coating material. When the two are inconsistent, mark it as a feature judgment conflict event and obtain the grasping abnormal timestamp and the workstation location information of the robot arm when the feature judgment conflict event occurs. S1053. Based on the aforementioned features, determine the type of conflict event, capture the abnormal timestamp and workstation location information, generate an abnormality type judgment result containing the cause of the abnormality, the location and time of occurrence, and compile it into a structured review report.

[0068] In the above steps, the specific time period before and after the occurrence of the grasping anomaly refers to the interval from a certain period before the occurrence of the grasping anomaly to a certain period after the occurrence of the anomaly; the fused feature is the comprehensive feature formed in step S103, which includes static visual features and coating material distribution characteristics; the static visual features refer to the visually observable features such as the shape, color, and pattern of the lottery bag surface; the coating material distribution characteristics reflect the material composition and distribution state characteristics of the scratch-off layer coating; the independent analysis is to determine the state of the lottery bag based on the static visual features and the coating material distribution characteristics respectively; the judgment result based on the static visual features is the analysis result obtained based on the static visual features; the judgment result based on the coating material distribution characteristics is the analysis result obtained based on the coating material distribution characteristics; the feature judgment conflict event refers to the situation where the judgment result based on the static visual features is inconsistent with the judgment result based on the coating material distribution characteristics; the grasping anomaly timestamp records the specific time when the grasping anomaly occurred; the workstation location information refers to the specific working position of the robot arm; the anomaly type judgment result includes information such as the cause of the anomaly, the location and time of occurrence; the structured review report is a report formed by organizing the anomaly type judgment results according to a certain structure. In this embodiment, firstly, step S1051 involves retrospectively capturing the fusion features of the lottery package within a specific time period before and after the anomaly occurred, such as the fusion features from 10 seconds before the anomaly occurred to 10 seconds after. Static visual features and coating material distribution characteristics are separated from the fusion features, and a classification algorithm, such as a support vector machine algorithm, is used to independently classify these two types of features. For static visual features, key information such as the shape and color of the lottery package is extracted and input into the classification model to obtain a judgment result based on the static visual features, such as determining that the lottery package belongs to type B. For coating material distribution characteristics, information such as the material composition and distribution state of the scratch-off layer coating is extracted and input into the classification model to obtain a judgment result based on the coating material distribution characteristics, such as determining that the lottery package belongs to type C.

[0069] Secondly, in step S1052, the judgment result based on static visual features is compared with the judgment result based on the distribution characteristics of the coating material. If the two are different, such as the static judgment being type B while the coating material judgment is type C, it is marked as a feature judgment conflict event. At the same time, the capture anomaly timestamp when the event occurred is obtained through the system's time recording module, for example, down to a specific year, month, day, hour, minute, and second. The position information of the robot arm's workstation is obtained through the position sensing device, such as workstation number 3.

[0070] Finally, in step S1053, the type of conflict event is determined based on the characteristics, possible causes of the anomaly are analyzed, and an anomaly type judgment result is generated by combining the captured anomaly timestamp and workstation location information. This clarifies whether the anomaly is caused by static visual feature recognition deviation or an error in analyzing the distribution characteristics of the coating material, as well as the specific location and time of occurrence. This information is then compiled into a structured review report according to a preset structure, such as basic anomaly information, cause analysis, and location and time of occurrence. The report is clear, well-organized, and easy for relevant personnel to view and process.

[0071] In a practical application, an automated sorting system at a lottery processing center experienced an anomaly in its robotic arm's grasping operation. The system reviewed the fused features of lottery packages from 15 seconds before and after the anomaly, extracting static visual features and coating material distribution characteristics, and classifying them independently. Based on the static visual features, the lottery package was classified as type A, while based on the coating material distribution characteristics, it was classified as type D. This discrepancy was marked as a feature judgment conflict event. The timestamp of the grasping anomaly was subsequently obtained as 14:30:20 on May 10, 2024, and the robotic arm's location was station number 5. Based on this information, the analysis suggested that the judgment deviation was likely caused by external interference during the acquisition of the coating material distribution characteristics. An anomaly type judgment result was generated and compiled into a structured review report containing the cause of the anomaly, its location, and time, for technical personnel to reference and handle.

[0072] In the overall scheme of step S105 above, by retrospectively fusing features and performing independent classification and comparison, conflicts between static and dynamic feature judgments can be promptly identified. Combined with the time and location information of the anomaly, the anomaly type can be accurately determined and a standardized report generated. This helps to quickly locate the cause of the anomaly, provides a clear basis for technicians to handle problems, effectively improves the efficiency and accuracy of anomaly handling in the sorting system, and ensures the smooth progress of subsequent sorting work.

[0073] As an alternative solution (i.e., obtaining the judgment result of the static visual features, the judgment result based on the distribution characteristics of the coating material, or the judgment result based on the optical property parameters of the transparent heat-shrink film), step S105 may specifically include: The review process traces back the fusion features of lottery packages within a specific time period before and after the anomaly occurred. Analysis is performed based on the static visual features, the distribution characteristics of the coating material, and the optical properties of the transparent heat-shrink film within these fusion features. This yields judgment results based on the static visual features, the distribution characteristics of the coating material, and the optical properties of the transparent heat-shrink film. These judgment results are compared, and if at least two judgment results are inconsistent, they are marked as feature judgment conflict events. The timestamp of the capture anomaly and the location information of the robot's workstation at the time of the feature judgment conflict event are obtained. Based on the type of feature judgment conflict event, the category of conflicting features involved, the timestamp of the capture anomaly, and the workstation location information, an anomaly type judgment result containing the cause of the anomaly, its location, and time is generated and compiled into a structured review report.

[0074] Since the alternative solution is similar to the one described above, except that it adds a judgment result based on the optical property parameters of the transparent heat-shrink film, this application will not repeat the explanation. In general, after adding the judgment and analysis of the optical property parameters of the transparent heat-shrink film, the process of generating anomaly type judgment results and forming a review report further incorporates the assessment of the integrity of the outer packaging, building upon the original dual judgment based on static visual features and the distribution characteristics of the coating material. Specifically, the system will generate an additional judgment result based on the optical property parameters of the transparent heat-shrink film. When at least two of these three judgment results are inconsistent, it is marked as a feature judgment conflict event. When analyzing the cause of the anomaly, the specific feature categories involved in the conflict will be comprehensively considered (e.g., a conflict between the coating material and heat-shrink film parameters, or a conflict between static visual features and heat-shrink film parameters), thereby enabling a more accurate determination of whether the anomaly stems from an inherent quality problem with the lottery ticket itself or a defect in the outer packaging, thus generating a more comprehensive and accurate anomaly type judgment result and a structured review report.

[0075] The following is a complete embodiment for steps S101 to S105: like Figure 3 As shown, suppose a lottery marketing center undertakes the sorting task of a batch of Class A lottery packages. First, at the beginning of the production line, high-speed cameras capture images of the lottery packages moving at high speed on the conveyor belt, obtaining clear and continuous video stream images. Subsequently, super-resolution reconstruction technology is applied to these images, and a dynamic inter-frame alignment algorithm is used to accurately align adjacent frames. Then, through multi-scale feature fusion, the details lost on the surface of the lottery packages due to blurring caused by high-speed motion are restored, successfully generating a high-resolution image sequence.

[0076] As an example, static visual features such as shape, color, and pattern on the surface of the lottery package are extracted from the sequence. Simultaneously, a multispectral imaging device is used to excite the scratch-off layer of the lottery package, acquiring its fluorescence spectral response characteristics. Spectral decomposition is then performed to obtain the distribution characteristics of the coating material. These two features are combined to form a multimodal fusion feature, which is then matched with a pre-stored standard feature library to obtain preliminary sorting results. During the sorting process, the robotic arm's grasping actions are continuously monitored, recording its start time, execution trajectory, and completion time. The time interval and trajectory deviation are calculated as temporal consistency indicators. When the preliminary sorting results are at the sorting threshold or the temporal consistency of the grasping actions is abnormal, a review procedure is initiated. For example, after a grasping anomaly occurs, the fusion features of the lottery package from 20 seconds before to 20 seconds after the anomaly are traced back and analyzed based on both static visual features and the distribution characteristics of the coating material. If the judgment results based on the static visual features are inconsistent with the judgment results based on the distribution characteristics of the coating material, this is marked as a feature judgment conflict event. Obtain the timestamp of the event when it occurred, such as 10:25:30 on October 15, 2024, as well as the location information of the robot arm at workstation number 7. Analyze the data to generate an anomaly type judgment result and form a structured review report.

[0077] As another example, static visual features such as shape, color, and pattern on the surface of the lottery package are extracted from the sequence. Simultaneously, a multispectral imaging device is used to excite the scratch-off layer of the lottery ticket, acquiring its fluorescence spectral response characteristics. Spectral decomposition is then used to obtain the distribution characteristics of the coating material. Furthermore, optical sensors are used to acquire the transmittance distribution characteristics and surface refractive properties of the transparent heat-shrinkable film on the outer packaging of the lottery package. These three features—static visual features, coating material distribution characteristics, and heat-shrinkable film optical properties—are combined to form a multimodal fusion feature. This feature is then matched with a pre-stored standard feature library to obtain preliminary sorting results. During the sorting process, the robotic arm's gripping actions are continuously monitored. For example, after a gripping anomaly occurs, the fusion features of the lottery package from 20 seconds before to 20 seconds after the anomaly are reviewed. Analysis is performed based on static visual features, coating material distribution characteristics, and heat-shrinkable film optical properties. It is found that the judgment results based on the coating material distribution characteristics are inconsistent with those based on the heat-shrinkable film optical properties, while the judgment results based on the static visual features are normal. This is marked as a feature judgment conflict event. The system obtains the capture anomaly timestamp and mechanical / manual position information when the event occurred. After analysis, it is determined that the cause of the anomaly is a defect in the heat shrink film of the outer packaging, but the lottery ticket itself is intact. The system generates a judgment result for the anomaly of the outer packaging and forms a detailed structured review report.

[0078] The automatic sorting and verification method for instant lottery tickets based on super-resolution vision provided in this application clearly presents the details of the lottery ticket package through super-resolution reconstruction, thereby improving the accuracy of feature extraction.

[0079] Figure 4 This application provides a schematic diagram of a specific implementation of an automatic sorting and verification system for instant lottery tickets based on super-resolution vision, referring to... Figure 4 The system may include: The acquisition module 41 is used to acquire clear, continuous video stream images of the lottery package in a high-speed motion state; The reconstruction module 42 is used to perform super-resolution reconstruction processing on the continuous video stream images. By using dynamic inter-frame alignment and multi-scale feature fusion, it restores the detail information lost on the surface of the lottery package due to motion blur, so as to generate a high-resolution image sequence. The decomposition module 43 is used to extract static visual features of the surface of the lottery package from the high-resolution image sequence, simultaneously acquire the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package, perform spectral decomposition processing on the fluorescence spectral response features to obtain the distribution characteristics of the coating material of the first lottery ticket, combine the static visual features and the distribution characteristics of the coating material to form a multimodal fusion feature, and match and compare the fusion feature with the pre-stored standard feature library to obtain the preliminary sorting results; Monitoring module 44 is used to continuously monitor the timing consistency of the robotic arm's gripping actions during the sorting process. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated. The generation module 45 is used to trace back the fusion features of the lottery package within a specific time period before and after the occurrence of the anomaly. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the module combines the anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

[0080] The automatic sorting and verification system for instant lottery tickets based on super-resolution vision in this application embodiment is used to implement the aforementioned automatic sorting and verification method for instant lottery tickets based on super-resolution vision. Therefore, the specific implementation of the automatic sorting and verification system for instant lottery tickets based on super-resolution vision can be found in the embodiment section of the automatic sorting and verification method for instant lottery tickets based on super-resolution vision above. The specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0081] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-described automatic sorting and verification method for instant lottery tickets based on super-resolution vision.

[0082] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described methods for automatic sorting and verification of instant lottery tickets based on super-resolution vision.

[0083] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0084] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the automatic sorting and verification method for instant lottery tickets based on super-resolution vision.

[0085] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0086] The foregoing has provided a detailed description of the automatic sorting and verification method, system, electronic device, and storage medium for instant lottery tickets based on super-resolution vision, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for automatic sorting and verification of whole packages of instant lottery tickets based on super-resolution vision, characterized in that, include: Acquire clear, continuous video stream images of lottery packages in high-speed motion; Super-resolution reconstruction processing is performed on the continuous video stream images. Through dynamic inter-frame alignment and multi-scale feature fusion, the detailed information lost on the surface of the lottery package due to motion blur is restored to generate a high-resolution image sequence. Static visual features of the lottery package surface are extracted from the high-resolution image sequence. Simultaneously, the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package are obtained. The fluorescence spectral response features are subjected to spectral decomposition to obtain the distribution characteristics of the coating material of the first lottery ticket. The static visual features and the distribution characteristics of the coating material are combined to form a multimodal fusion feature. The fusion feature is matched and compared with a pre-stored standard feature library to obtain preliminary sorting results. During the sorting process, the timing consistency of the robotic arm's gripping actions is continuously monitored. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated. The review process involves backtracking and capturing the fusion features of lottery packages within a specific time period before and after the anomaly occurs. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the process combines the captured anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

2. The method according to claim 1, characterized in that, After performing super-resolution reconstruction processing on the continuous video stream images, and recovering the detail information lost on the surface of the lottery package due to motion blur through dynamic inter-frame alignment and multi-scale feature fusion to generate a high-resolution image sequence, the method further includes: Obtain the optical properties of the transparent heat-shrink film used for lottery ticket packaging, including its transmittance distribution characteristics and surface refractive properties. The method further includes: By combining the static visual features, the distribution characteristics of the coating material of the first lottery ticket, and the optical properties of the transparent heat-shrink film, a multimodal fusion feature is formed. The multimodal fusion features are matched and compared with a pre-stored standard feature library. The quality of the entire package of lottery tickets is judged based on the coating status of the first lottery ticket and the integrity of the outer packaging, and a preliminary sorting result is obtained.

3. The method according to claim 1, characterized in that, Obtain the fluorescence spectral response characteristics of the scratch-off layer coating corresponding to the lottery ticket on the homepage of the lottery package. Perform spectral decomposition processing on the fluorescence spectral response characteristics to obtain the distribution characteristics of the coating material of the first lottery ticket. Combine the static visual characteristics with the distribution characteristics of the coating material to form a multimodal fusion feature, including: Based on the positioning information of the high-resolution image sequence, the multispectral optical imaging system is controlled to excite the scratch-off layer coating of the first lottery ticket using multiple light sources of different wavelengths in a time-division polling manner. The multispectral optical imaging system is used to acquire fluorescence images of the excited scraped layer in multiple spectral bands, and a continuous spectral reflectance curve for each pixel is generated based on the fluorescence images. The spectral reflectance curve is decomposed into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum, and the contribution of each spectral component to the pixel is calculated. Based on the contribution analysis, the continuous variation characteristics of the standard spectrum of the coating in spatial distribution are analyzed to identify whether there are uneven thickness or abnormal coverage areas in the scratched layer as the coating anomaly identification result. At the same time, based on the temporal variation of multiple frames in the high-resolution image sequence, the motion posture change characteristics of the lottery bag are analyzed. The static visual features, the results of the film overlay anomaly recognition, and the motion posture change features are fused in a multimodal manner to form a fused feature.

4. The method according to claim 3, characterized in that, The spectral reflectance curve is decomposed into a linear combination of a predefined standard coating spectrum, a paper substrate spectrum, and a printing ink spectrum. The contribution of each spectral component to a pixel is calculated, including: For each pixel of the scratch-off layer of the first lottery ticket in the lottery package, a corresponding spectral reflectance curve is established. The spectral reflectance curve is composed of the reflectance intensity values ​​of multiple spectral bands and is expressed as a weighted sum of the corresponding components of the standard spectral of the coating, the corresponding components of the spectral of the paper substrate, and the corresponding components of the spectral of the printing ink. By minimizing the difference between the spectral reflectance curve and the weighted sum, the weighting coefficients of the corresponding components of the coated standard spectrum, the corresponding components of the paper substrate spectrum, and the corresponding components of the printing ink spectrum are solved. The weighting coefficients are used as the contribution of each spectral component to the corresponding pixel.

5. The method according to claim 3, characterized in that, Based on the contribution analysis, the continuous variation characteristics of the standard spectrum of the coating in spatial distribution are analyzed to identify whether there are uneven thicknesses or abnormal coverage areas in the scratched layer as coating anomaly identification results. Simultaneously, based on the temporal changes of multiple frames in the high-resolution image sequence, the motion posture change characteristics of the lottery bag are analyzed, including: The contribution of the corresponding component of the coating standard spectrum in each pixel is arranged according to its actual position. The change of contribution of adjacent pixels is detected and a continuous numerical sequence is recorded. The spatial continuity variation characteristics of the coating standard spectrum are determined by the fluctuation amplitude and change frequency of the continuous numerical sequence. Based on the continuous change characteristics, regions where the contribution of multiple consecutive pixels changes beyond a preset range and where the overall contribution of the region differs significantly from the surrounding area are identified as areas with uneven thickness of the scraping layer. The region with a local contribution value of zero or far exceeding the normal range, and which forms an abrupt boundary with the surrounding area, is identified as an abnormal coverage area of ​​the scraped layer. The location information of the uneven thickness area and the abnormal coverage area are integrated to form the film abnormality identification result. Fixed markers on the surface of the lottery bag are selected from the high-resolution image sequence, and the coordinate information of the fixed markers on the surface of the lottery bag is obtained by tracking the position changes of the markers in each frame of the image. Based on the coordinate information, the relative change of the coordinates of the marker points in adjacent frames is calculated, and the rotation angle and displacement distance of the lottery package per unit time are determined based on the relative change. According to the change law of the rotation angle and displacement distance of the lottery package, the motion posture change characteristics of the lottery package are formed.

6. The method according to claim 1, characterized in that, The detailed information includes category identification and micro-anti-counterfeiting patterns; the super-resolution reconstruction processing of the continuous video stream images, through dynamic inter-frame alignment and multi-scale feature fusion, restores the detailed information lost on the surface of the lottery package due to motion blur, in order to generate a high-resolution image sequence, including: Multiple temporally consecutive frames are selected from the continuous video stream as input frames. The scale-invariant feature transform algorithm is used to extract feature points of each frame in the input frame group, and the fast nearest neighbor search matching algorithm is used to calculate the correspondence between the feature points of different frames. Based on the feature points and their correspondence, the positional offset between each pair of feature points is calculated, and an affine transformation algorithm is used to perform geometric transformation on each frame image in the input frame group based on the positional offset, so that the relative positions of the lottery package in multiple frames remain consistent. The grayscale values ​​of each pixel in the aligned multi-frame images are weighted and averaged to generate a preliminary fused image. The weights of the weighted average calculation are dynamically adjusted according to the signal-to-noise ratio of each pixel in different frames. The preliminary fused image is input into a multi-layer processing structure, and multi-scale features are extracted through convolution operations to obtain feature maps containing different levels of abstraction. The feature map is upsampled to increase its resolution and gradually restore spatial details. By using skip connections, low-level detailed features are fused with upsampled high-level semantic features to generate enhanced feature representations; Based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by dynamic trailing are reconstructed layer by layer, the text outline of the category identification and the detailed features of the micro anti-counterfeiting pattern are restored, and a high-resolution image sequence is output.

7. The method according to claim 6, characterized in that, Based on the enhanced feature representation, the edge texture and high-frequency information of the lottery package surface damaged by dynamic trailing are reconstructed layer by layer, restoring the text outline of the category identifier and the detailed features of the micro-anti-counterfeiting pattern, and outputting a high-resolution image sequence, including: The basic features of edge texture and the detailed features of high-frequency information are separated from the enhanced feature representation; Based on the basic features of the edge texture, in the area covered by the dynamic trailing shadow, missing edge segments and broken contour lines on the surface of the lottery package are supplemented, and the detailed features of the high-frequency information are filled in within the edge texture framework to restore the texture's light and dark transitions and subtle undulations. Based on the text-related feature components in the enhanced feature representation, the contrast of the stroke edges of the characters is strengthened and broken parts are repaired in the category identification area to form a continuous and closed character outline. Based on the feature points with a size smaller than a preset threshold in the enhanced feature representation, adjacent feature points are connected in the miniature anti-counterfeiting pattern area according to the pattern arrangement rule to form a complete pattern unit and supplement the connection structure between units. The restored edge texture, high-frequency information, category identification text outline, and micro anti-counterfeiting pattern are integrated into a single high-resolution image. All the generated single high-resolution images are arranged in chronological order to form a high-resolution image sequence.

8. The method according to claim 1, characterized in that, The system traces back the fusion features of lottery packages within a specific time period before and after the anomaly occurs. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, it combines the anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report, including: The verification procedure is used to trace back and capture the fusion features of lottery packages within a specific time period before and after the anomaly occurred. The static visual features and the distribution characteristics of the coating material in the fusion features are analyzed to obtain the judgment results of the static visual features and the judgment results based on the distribution characteristics of the coating material. The judgment results of the static visual features are compared with the judgment results based on the distribution characteristics of the coating material. When the two are inconsistent, they are marked as feature judgment conflict events. The grabbing anomaly timestamp and the workstation location information of the robot arm when the feature judgment conflict event occurs are obtained. Based on the aforementioned features, the type of conflict event is determined, abnormal timestamps and workstation location information are captured, an abnormality type determination result containing the cause of the abnormality, the location and time of occurrence is generated, and compiled into a structured review report.

9. The method according to claim 1, characterized in that, During the sorting process, the timing consistency of the robotic arm's gripping actions is continuously monitored. When the preliminary sorting result is at a critical sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated, including: During the sorting process, the start time, execution trajectory, and completion time of each grasping action of the robotic arm are recorded. Based on the start time, execution trajectory, and completion time, the time interval and trajectory deviation are calculated as timing consistency indicators. Set an upper and lower limit for the sorting threshold. When the similarity metric value corresponding to the preliminary sorting result is between the upper and lower limits, it is determined to be at the sorting threshold threshold. When the timing consistency index exceeds the preset tolerance range, it is determined that the timing consistency of the grabbing action is abnormal. When either the sorting threshold threshold or the timing consistency anomaly occurs, a review instruction is triggered, and the review process is started.

10. An automatic sorting and verification system for instant lottery tickets in whole packages based on super-resolution vision, characterized in that, include: The acquisition module is used to acquire clear, continuous video stream images of lottery packages in high-speed motion. The reconstruction module is used to perform super-resolution reconstruction processing on the continuous video stream images. By using dynamic inter-frame alignment and multi-scale feature fusion, it restores the detail information lost on the surface of the lottery package due to motion blur, so as to generate a high-resolution image sequence. The decomposition module is used to extract static visual features of the surface of the lottery package from the high-resolution image sequence, simultaneously acquire the fluorescence spectral response features of the scratch-off layer coating corresponding to the lottery ticket on the first page of the lottery package, perform spectral decomposition processing on the fluorescence spectral response features to obtain the distribution characteristics of the coating material of the first lottery ticket, combine the static visual features and the distribution characteristics of the coating material to form a multimodal fusion feature, and match and compare the fusion feature with a pre-stored standard feature library to obtain preliminary sorting results; The monitoring module is used to continuously monitor the timing consistency of the robotic arm's gripping actions during the sorting process. When the preliminary sorting result is at the sorting threshold or the timing consistency of the gripping actions is abnormal, a review procedure is initiated. The generation module is used to retrospectively capture the fusion features of lottery packages within a specific time period before and after the occurrence of an anomaly. When the judgment result of the static visual features is inconsistent with the judgment result based on the distribution characteristics of the coating material, the module combines the captured anomaly timestamp and workstation location information to generate an anomaly type judgment result and form a review report.

Images