Lottery store violation abnormity identification method and system

Abstract

The invention provides a lottery store violation abnormity identification method and system, and relates to the technical field of image identification, and the method comprises the steps: collecting multi-source heterogeneous image data in real time through an edge computing device; preprocessing the collected multi-source heterogeneous image data to form a stereo image data set containing space, depth and thermodynamic information; constructing a multi-task deep learning model, wherein the multi-task deep learning model comprises a face recognition module, a behavior analysis module and an environment detection module; inputting the three-dimensional image data set into the trained multi-task deep learning model, and executing face recognition, behavior analysis and environment detection tasks at the same time; based on an execution result of the multi-task deep learning model, generating a violation score by applying a fuzzy comprehensive evaluation method, and weighting by considering historical violation records; and generating a lottery store site investigation plan based on the violation scoring result.

Description

Technical Field

[0001] The present invention relates to image recognition technology, and in particular to a method and system for identifying irregularities in lottery stores. Background Art

[0002] Traditional methods of identifying violations and anomalies in lottery stores mainly rely on manual inspections and simple video surveillance systems, which have many problems such as incomplete coverage, limited recognition capabilities, single data source, rigid evaluation methods, neglect of historical records, poor real-time performance, and lack of intelligent troubleshooting.

[0003] The manual inspection method has problems such as high labor costs, limited coverage, and poor real-time performance. Inspectors cannot monitor every lottery sales point 24 hours a day, resulting in many violations that may occur without being discovered during the inspection intervals.

[0004] Although simple video surveillance systems have improved the real-time nature and coverage of supervision to a certain extent, their analytical capabilities are limited. Such systems are usually only capable of basic image acquisition and storage, and lack intelligent abnormal behavior recognition capabilities. Supervisors need to manually review a large amount of video recordings, which is inefficient and easy to miss important information.

[0005] Some slightly more advanced systems have begun to introduce basic image recognition techniques, such as face recognition or simple action detection. However, these systems are often designed for a single task and cannot fully consider the complex situations in the lottery sales environment. Summary of the invention

[0006] The embodiments of the present invention provide a lottery store violation anomaly identification method and system, which can solve the problems in the prior art.

[0007] According to a first aspect of the embodiments of the present invention, Provides a method for identifying irregularities in lottery stores, including: Install high-resolution RGB cameras, depth cameras, and thermal imaging cameras in lottery stores, and use edge computing devices to collect multi-source heterogeneous image data in real time; pre-process the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth, and thermal information; Construct a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module, and use a labeled data set to train and optimize the multi-task deep learning model; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and judges suspected minors by age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The stereo image dataset is input into the trained multi-task deep learning model, and face recognition, behavior analysis and environmental detection tasks are performed simultaneously. Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting. Based on the violation score results, an on-site inspection plan for lottery stores is generated.

[0008] In an optional embodiment, Using edge computing devices to collect multi-source heterogeneous image data in lottery stores in real time, the multi-source heterogeneous image data including RGB image data, depth image data and thermal imaging image data; Applying an improved conditional generative adversarial network to the RGB image data for image enhancement, wherein the generator of the conditional generative adversarial network adopts a U-Net structure and introduces a channel attention mechanism in the jump connection, and the discriminator adopts a PatchGAN structure, and optimizes network parameters by combining L1 loss, adversarial loss and perceptual loss function; The enhanced RGB image data, depth image data and thermal imaging image data are registered using an end-to-end registration framework based on deep learning. The end-to-end registration framework includes a feature extraction network and a transformation estimation network. The feature extraction network uses a twin structure to process image data of different modalities respectively. The transformation estimation network uses a multi-layer perceptron structure to output affine transformation parameters and optimizes the registration accuracy through a multi-modal similarity metric loss function. The registered image data is input into a multi-scale feature fusion network, which adopts an improved feature pyramid network structure, introduces a cross-modal self-attention mechanism to enhance information interaction between different modalities, and dynamically adjusts the fusion weights of different modal features through an adaptive feature aggregation module; The output of the multi-scale feature fusion network is converted into a unified tensor format to generate a stereo image dataset containing spatial, depth, and thermal information.

[0009] In an optional embodiment, Construct a multi-task deep learning model, the multi-task deep learning model includes a face recognition module, a behavior analysis module and an environment detection module, and the steps of training and optimizing the multi-task deep learning model using a labeled data set include: Constructing a multi-task deep learning model, the multi-task deep learning model includes a shared feature extraction backbone network and multiple task branch networks, the shared feature extraction backbone network adopts an improved EfficientNet-B4 structure, introduces dynamic convolution and Squeeze-and-Excitation modules, and generates feature maps of multiple scales through a multi-scale feature fusion mechanism; the multiple task branch networks include a face recognition module branch network, a behavior analysis module branch network, and an environment detection module branch network; Design a dynamic task weight allocation strategy, including building a task difficulty assessment module, dynamically adjusting task weights by monitoring the rate of change of training loss for each task, using the Softmax function to map task difficulty to a predetermined interval, and using the mapping result as a scaling factor for task weights; Formulate an uncertainty-based multi-task loss fusion method, introduce a learnable uncertainty parameter for each task, divide the loss of each task by the square of the corresponding uncertainty parameter, and add the logarithm of the product of all uncertainty parameters to obtain the total loss function; Implement data augmentation strategies, including geometric transformation and color jittering of the annotated dataset, and use the fast gradient sign method to generate adversarial samples, which are then mixed with the original samples to form an enhanced annotated dataset. The multi-task deep learning model is trained using the enhanced labeled data set. During the training process, the weight of each task is adjusted based on a dynamic task weight allocation strategy, and the total loss is calculated using an uncertainty-based multi-task loss fusion method; The multi-task deep learning model parameters are optimized through the back-propagation algorithm, and the uncertainty parameters and task weights are updated simultaneously until the multi-task deep learning model converges or reaches the preset training rounds.

[0010] In an optional embodiment, The face recognition module uses the improved RetinaFace algorithm to perform face detection and uses the SE-ResNet-based age estimation model to estimate age. The steps of judging suspected minors by age threshold include: Based on a stereoscopic image dataset, face detection is performed using an improved RetinaFace algorithm, wherein the improved RetinaFace algorithm includes a channel attention module, a multi-scale context module and an adaptive sample selection strategy in a feature pyramid network, wherein the channel attention module enhances the perception of faces of different scales through adaptive average pooling and maximum pooling operations, the multi-scale context module captures multi-scale context information through different convolution kernel sizes, and the adaptive sample selection strategy dynamically adjusts the ratio of positive and negative samples according to the category probability; Inputting the detected face image into an age estimation model based on SE-ResNet, wherein the SE-ResNet model includes a Squeeze-and-Excitation module; Based on the output of the SE-ResNet age estimation model, a soft threshold strategy is used to judge minors, wherein the soft threshold strategy converts the age prediction value into the probability of a minor through a Sigmoid function, makes a final judgment based on the set probability threshold, and outputs the minor identification result.

[0011] In an optional embodiment, The behavior analysis module, based on the behavior recognition method of the improved spatiotemporal graph convolutional network, determines whether there is a ticket purchase behavior of underage students, including the following steps: Dynamically generate the connection relationship between the human body feature points in the stereoscopic image data set using an adaptive graph construction module to form an adaptive graph structure; wherein the adaptive graph construction module includes an edge predictor, and the edge predictor calculates the edge weight according to the input feature through a multi-layer neural network; Inputting the adaptive graph structure into a multi-scale spatiotemporal convolution module, wherein the multi-scale spatiotemporal convolution module comprises a graph convolution layer and a temporal convolution layer with different convolution kernel sizes; Applying a spatiotemporal attention mechanism on the output features of the multi-scale spatiotemporal convolution module, including a spatial attention submodule and a temporal attention submodule; The weighted features of the spatial attention submodule and the temporal attention submodule are input into the fully connected layer for feature fusion and dimensionality reduction; Perform behavior classification based on the fused features and output the probability distribution of behavior categories; According to the predefined behavior category mapping table, the identified behavior category probability distribution is matched with the ticket purchasing behavior of young students, and the behavior recognition result is output.

[0012] In an optional embodiment, The environment detection module uses the improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in the lottery store, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items. The steps include: Inputting the stereo image dataset into an improved Mask R-CNN model, wherein the improved Mask R-CNN model includes a backbone network, a feature enhancement network, an improved region proposal network, and an improved ROI Head; Extracting multi-level features through the backbone network; Inputting the multi-level features into the feature enhancement network, the feature enhancement network includes a multi-scale feature fusion module, an attention enhancement module and a residual dense block, wherein the multi-scale feature fusion module fuses features of different levels, the attention enhancement module includes a channel and spatial attention mechanism, and the residual dense block uses the multi-level features for feature extraction; Inputting the output features of the feature enhancement network into the improved region proposal network, wherein the improved region proposal network adopts a dense anchor box generation strategy to generate candidate regions including standard anchor boxes and dense anchor boxes; Inputting the candidate region and the features enhanced by the feature enhancement network into the improved ROI Head, wherein the improved ROI Head includes an adaptive feature pooling layer; introducing a contrastive learning module after the improved ROI Head, wherein the contrastive learning module improves the ability to distinguish similar objects through a contrastive learning strategy; The improved Mask R-CNN model is trained using an improved loss function, wherein the improved loss function includes Focal Loss and IoU-balanced Loss. The trained model is inferred to obtain instance segmentation results and recognition results of illegal items, wherein the illegal items include electric bicycles and high-power electrical appliances.

[0013] In an optional embodiment, The steps to introduce contrastive learning strategies and improve the ability to distinguish similar objects include: Inputting the historical image data set into a neural network model including a backbone network and a projection head; extracting feature representations of the historical electrical appliance images through the backbone network; inputting the feature representations into the projection head to obtain normalized feature projections; Performing data enhancement processing on the historical image data to generate enhanced historical appliance image data, wherein the data enhancement processing includes random rotation, random cropping, color jittering, Gaussian blurring, and adding noise; inputting the enhanced historical appliance image data into the neural network model to obtain feature projections of the enhanced image; calculating a supervised contrast loss based on the normalized feature projections; calculating a self-supervised contrast loss based on the feature projections of the enhanced image; Combining the supervised contrastive loss and the self-supervised contrastive loss, we get the joint contrastive learning loss; Optimizing and training the neural network model based on the joint contrastive learning loss; Use the trained neural network model to extract features and project the stereo image dataset; Based on feature extraction and projection, the similarity between the stereo image dataset and the known electrical appliance categories is calculated, the category of the stereo image dataset is determined, and the electrical appliance recognition result is obtained.

[0014] In an optional embodiment, Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and the steps of weighting by considering historical violation records include: A fuzzy comprehensive evaluation model is constructed, wherein the fuzzy comprehensive evaluation model includes a target layer, a criterion layer, and an indicator layer, wherein the target layer is a violation score, the criterion layer includes age factors, behavior factors, and item factors, and the indicator layer includes a minor indicator, an adult indicator, a purchase behavior indicator, a retention behavior indicator, an interaction behavior indicator, and an illegal item indicator; a machine learning algorithm is used to determine the weight of each layer in the fuzzy comprehensive evaluation model, including using historical data to train a decision tree model, generating a feature importance matrix, calculating a feature vector, and performing cross-validation to obtain a weight vector; Design membership functions to convert the output of the multi-task deep learning model into fuzzy sets, wherein the membership functions include age membership functions, behavior membership functions, and item membership functions; based on the membership functions, convert the recognition results into fuzzy sets; construct a single factor evaluation matrix, wherein the single factor evaluation matrix includes age evaluation matrix, behavior evaluation matrix, and item evaluation matrix; The fuzzy comprehensive evaluation results are calculated using the weight vector and the single factor evaluation matrix; Combine the fuzzy comprehensive evaluation results to obtain the total evaluation matrix; Multiply the weight vector by the total judgment matrix to obtain the final judgment result; The final judgment result and the preset violation grade score are weighted and summed to obtain the initial violation score; Obtain historical violation records and calculate weighting factors based on the number of historical violations; The initial violation score is weighted using the weighting factor to obtain a final violation score; Output the final violation score as the evaluation result of the lottery store’s violation behavior.

[0015] According to a second aspect of the embodiments of the present invention, Provides a lottery store violation and anomaly identification system, including: The first unit is used to install high-resolution RGB cameras, depth cameras and thermal imaging cameras in lottery stores, and use edge computing devices to collect multi-source heterogeneous image data in real time; pre-process the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth and thermal information; The second unit is used to build a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module. The multi-task deep learning model is trained and optimized using a labeled data set; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and determines suspected minors through an age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The third unit is used to input the stereoscopic image dataset into the trained multi-task deep learning model, and simultaneously perform face recognition, behavior analysis, and environmental detection tasks; based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting; based on the violation score results, a lottery store on-site inspection plan is generated.

[0016] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0017] This paper realizes the comprehensive capture of scene information through the collection and processing of multi-source heterogeneous image data, providing a rich data foundation for subsequent analysis tasks. Secondly, the improved conditional generative adversarial network performs well in image enhancement and can effectively improve the visual effect and information content of low-quality images. Thirdly, the end-to-end deep learning registration framework improves the accuracy and efficiency of multimodal image registration, laying the foundation for subsequent feature fusion. Finally, the design of the multi-scale feature fusion network, especially the introduction of the cross-modal self-attention mechanism and the adaptive feature aggregation module, realizes the effective integration of different modal information and generates a stereo image dataset containing rich semantic information.

[0018] The adaptive graph construction module adopted in this paper can dynamically adjust the connection relationship between the key points of the human body, improving the model's adaptability to different postures and actions. Secondly, the multi-scale spatiotemporal convolution module effectively captures the spatial and temporal features of different scales, enhancing the model's ability to understand complex actions. The spatiotemporal attention mechanism further highlights the key spatial areas and time periods, improving the accuracy of recognition. Finally, through the predefined behavior category mapping table, the system can flexibly adapt to the ticket purchasing behavior patterns of young students in different scenarios. This comprehensive optimization method not only improves the accuracy and robustness of behavior recognition, but also has good generalization ability and can be applied to various complex practical scenarios.

[0019] The violation scoring method based on fuzzy comprehensive evaluation and weighted historical violation records adopted in this paper has many beneficial effects. First, it can comprehensively consider multiple influencing factors, including age, behavior, and items, making the evaluation results more objective and comprehensive. Secondly, by using fuzzy theory, this method can better deal with uncertainty and ambiguity in the real world, avoiding the limitations of traditional hard threshold methods. Furthermore, the introduction of machine learning algorithms to determine weights enables the evaluation model to adaptively adjust the importance of each factor based on historical data, improving the accuracy and adaptability of the model. In addition, the design of weighted historical violation records can impose more severe penalties on repeated violations, which is conducive to curbing repeated violations. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a method for identifying irregularities in lottery stores according to an embodiment of the present invention; Figure 2 The present invention is a schematic diagram of the structure of a lottery store violation anomaly identification system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0023] Figure 1 FIG. 1 is a flow chart of a method for identifying illegal anomalies in lottery stores according to an embodiment of the present invention. Figure 1 As shown, the method includes: Install high-resolution RGB cameras, depth cameras, and thermal imaging cameras in lottery stores, and use edge computing devices to collect multi-source heterogeneous image data in real time; pre-process the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth, and thermal information; Construct a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module, and use a labeled data set to train and optimize the multi-task deep learning model; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and judges suspected minors by age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The stereo image dataset is input into the trained multi-task deep learning model, and face recognition, behavior analysis and environmental detection tasks are performed simultaneously. Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting. Based on the violation score results, an on-site inspection plan for lottery stores is generated.

[0024] In an optional embodiment, The steps of preprocessing the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth and thermal information include: Using edge computing devices to collect multi-source heterogeneous image data in lottery stores in real time, the multi-source heterogeneous image data including RGB image data, depth image data and thermal imaging image data; Applying an improved conditional generative adversarial network to the RGB image data for image enhancement, wherein the generator of the conditional generative adversarial network adopts a U-Net structure and introduces a channel attention mechanism in the jump connection, and the discriminator adopts a PatchGAN structure, and optimizes network parameters by combining L1 loss, adversarial loss and perceptual loss function; The enhanced RGB image data, depth image data and thermal imaging image data are registered using an end-to-end registration framework based on deep learning. The end-to-end registration framework includes a feature extraction network and a transformation estimation network. The feature extraction network uses a twin structure to process image data of different modalities respectively. The transformation estimation network uses a multi-layer perceptron structure to output affine transformation parameters and optimizes the registration accuracy through a multi-modal similarity metric loss function. The registered image data is input into a multi-scale feature fusion network, which adopts an improved feature pyramid network structure, introduces a cross-modal self-attention mechanism to enhance information interaction between different modalities, and dynamically adjusts the fusion weights of different modal features through an adaptive feature aggregation module; The output of the multi-scale feature fusion network is converted into a unified tensor format to generate a stereo image dataset containing spatial, depth, and thermal information.

[0025] Exemplarily, the method of preprocessing the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth and thermal information involves multiple key steps and technical innovations. The method first uses edge computing devices to collect multi-source heterogeneous image data in lottery stores in real time, including RGB images, depth images and thermal imaging images. This multimodal data collection method can fully capture scene information and provide rich raw data for subsequent processing.

[0026] In the image enhancement stage, an improved conditional generative adversarial network (cGAN) is used to process RGB images. The generator of this network is based on the U-Net structure (U-type network structure), and the channel attention mechanism is introduced in the jump connection, which can effectively improve the accuracy of feature transfer. For example, for blurry RGB images collected under low light conditions, by introducing channel attention, the network can better capture and restore the detailed information of the image. The discriminator adopts the PatchGAN structure (image block generative adversarial network structure), which improves the sensitivity to local features by discriminating the local area of ​​the image. During the training process, the L1 loss, adversarial loss and perceptual loss functions are combined to optimize the network parameters. Specifically, the L1 loss is used to ensure the pixel-level consistency between the generated image and the true value, the adversarial loss makes the generated image more realistic, and the perceptual loss helps to maintain the high-level semantic features of the image. Through this multi-objective optimization strategy, enhanced images with higher quality and richer details can be generated.

[0027] Image registration is a key step in achieving multimodal data fusion. This method adopts an end-to-end registration framework based on deep learning, including a feature extraction network and a transformation estimation network. The feature extraction network adopts a twin structure to process RGB, depth and thermal imaging image data respectively. This structural design can simultaneously consider the features of different modal images and improve the accuracy of registration. For example, for the texture information in the RGB image and the temperature distribution information in the thermal image, the network can learn the correspondence between them. The transformation estimation network adopts a multi-layer perceptron structure and outputs affine transformation parameters. The registration accuracy is optimized by designing a multimodal similarity metric loss function, such as mutual information maximization or structural similarity index. This end-to-end learning method avoids the limitations of manually designed features in traditional methods and can adaptively learn the correspondence between different modalities.

[0028] In the multimodal feature fusion stage, an improved feature pyramid network (FPN) structure is adopted. This structure can effectively integrate feature information of different scales and improve the model's ability to recognize multi-scale targets. On this basis, a cross-modal self-attention mechanism is introduced to enhance the information interaction between different modalities. For example, for human body detection tasks, the appearance information in RGB images, the spatial information in depth images, and the temperature information in thermal images can complement each other and provide a more comprehensive feature representation. In addition, the fusion weights of different modal features are dynamically adjusted through an adaptive feature aggregation module. This dynamic weight adjustment mechanism can adaptively determine the importance of each modal feature according to the characteristics of the input data, thereby achieving more flexible and robust feature fusion.

[0029] Finally, the output of the multi-scale feature fusion network is converted into a unified tensor format to generate a stereo image dataset containing spatial, depth, and thermal information. This unified data format facilitates subsequent analysis and processing while retaining the rich information of multimodal data.

[0030] This paper realizes the comprehensive capture of scene information through the collection and processing of multi-source heterogeneous image data, providing a rich data foundation for subsequent analysis tasks. Secondly, the improved conditional generative adversarial network performs well in image enhancement and can effectively improve the visual effect and information content of low-quality images. Thirdly, the end-to-end deep learning registration framework improves the accuracy and efficiency of multimodal image registration, laying the foundation for subsequent feature fusion. Finally, the design of the multi-scale feature fusion network, especially the introduction of the cross-modal self-attention mechanism and the adaptive feature aggregation module, realizes the effective integration of different modal information and generates a stereo image dataset containing rich semantic information. This high-quality dataset can significantly improve the performance of subsequent tasks such as target detection and scene understanding, and has broad application prospects in the field of intelligent visual analysis.

[0031] In an optional embodiment, Construct a multi-task deep learning model, the multi-task deep learning model includes a face recognition module, a behavior analysis module and an environment detection module, and the steps of training and optimizing the multi-task deep learning model using a labeled data set include: Constructing a multi-task deep learning model, the multi-task deep learning model includes a shared feature extraction backbone network and multiple task branch networks, the shared feature extraction backbone network adopts an improved EfficientNet-B4 structure, introduces dynamic convolution and Squeeze-and-Excitation modules, and generates feature maps of multiple scales through a multi-scale feature fusion mechanism; the multiple task branch networks include a face recognition module branch network, a behavior analysis module branch network, and an environment detection module branch network; Design a dynamic task weight allocation strategy, including building a task difficulty assessment module, dynamically adjusting task weights by monitoring the rate of change of training loss for each task, using the Softmax function to map task difficulty to a predetermined interval, and using the mapping result as a scaling factor for task weights; Formulate an uncertainty-based multi-task loss fusion method, introduce a learnable uncertainty parameter for each task, divide the loss of each task by the square of the corresponding uncertainty parameter, and add the logarithm of the product of all uncertainty parameters to obtain the total loss function; Implement data augmentation strategies, including geometric transformation and color jittering of the annotated dataset, and use the fast gradient sign method to generate adversarial samples, which are then mixed with the original samples to form an enhanced annotated dataset. The multi-task deep learning model is trained using the enhanced labeled data set. During the training process, the weight of each task is adjusted based on a dynamic task weight allocation strategy, and the total loss is calculated using an uncertainty-based multi-task loss fusion method; The multi-task deep learning model parameters are optimized through the back-propagation algorithm, and the uncertainty parameters and task weights are updated simultaneously until the multi-task deep learning model converges or reaches the preset training rounds.

[0032] For example, building a multi-task deep learning model and using annotated datasets for training and optimization is a complex and sophisticated process involving multiple key steps and innovations. First, the core of building a multi-task deep learning model is to design a network architecture that can effectively handle multiple related tasks. The model includes a shared feature extraction backbone network and multiple task branch networks.

[0033] The shared feature extraction backbone network adopts the improved EfficientNet-B4 structure (Efficient Network B4 structure). As the basic structure, EfficientNet-B4 has demonstrated excellent performance and efficiency in multiple computer vision tasks. On this basis, a dynamic convolution mechanism is introduced to enable the network to dynamically adjust the convolution kernel parameters according to the input data. For example, for image scenes of different complexities, the network can adaptively select appropriate convolution operations to improve the flexibility and accuracy of feature extraction. In addition, the Squeeze-and-Excitation module (SE module) is introduced to enhance the network's attention to important features by explicitly modeling the interdependence between feature channels. Specifically, the module first performs global average pooling on the feature map, then learns the importance weights between channels through two fully connected layers, and finally applies these weights to the original feature map to achieve feature recalibration.

[0034] In order to make full use of multi-scale information, the network also introduces a multi-scale feature fusion mechanism. This mechanism generates feature maps of multiple scales by establishing connections between feature maps at different levels. For example, a shallow high-resolution feature map can be fused with a deep low-resolution feature map, which not only retains fine-grained spatial information but also contains high-level semantic information. This multi-scale feature representation is particularly important for tasks that simultaneously process objects of different scales.

[0035] Based on the backbone network, multiple task branch networks are designed, including face recognition module, behavior analysis module and environment detection module. Each branch network is optimized for a specific task. For example, the face recognition module may adopt a deep residual network structure and add an attention mechanism to capture key facial features; the behavior analysis module may combine a temporal convolutional network and a long short-term memory network to effectively model human action sequences; the environment detection module may use a feature pyramid network to process environmental objects of different scales.

[0036] In order to balance the learning process of multiple tasks, a dynamic task weight allocation strategy is designed. The core of this strategy is to build a task difficulty evaluation module, which dynamically adjusts the task weight by monitoring the training loss change rate of each task. In specific implementation, the loss change rate of each task within a certain number of training steps can be calculated and used as an indicator of task difficulty. Then, the Softmax function is used to map these difficulty indicators to a predetermined interval, such as [0.5, 2], as a scaling factor for the task weight. In this way, difficult tasks will receive higher weights, which will encourage the model to pay more attention to and learn from them.

[0037] In terms of loss function design, an uncertainty-based multi-task loss fusion method is adopted. Learnable uncertainty parameters are introduced for each task, and these parameters are dynamically adjusted during the training process. When calculating the loss, the loss of each task is divided by the square of the corresponding uncertainty parameter, and the logarithm of the product of all uncertainty parameters is added. This method can automatically balance the contribution of different tasks and avoid the tedious process of manually adjusting task weights. For example, if the uncertainty parameter of the face recognition task is large, its loss contribution will be relatively reduced, and the model will focus more on the optimization of other tasks.

[0038] Data augmentation is an important means to improve the generalization ability of the model. This method adopts a variety of data augmentation strategies, including geometric transformations (such as random cropping, rotation, flipping, etc.) and color jitter processing (such as random adjustment of brightness, contrast, and saturation). In addition, the fast gradient sign method is used to generate adversarial samples. Specifically, tiny perturbations are added to the original image. The direction of these perturbations is consistent with the direction of the model gradient, and the amplitude is controlled within a range that is difficult for the human eye to detect. These adversarial samples are mixed with the original samples to form an enhanced labeled data set. This method not only expands the diversity of training data, but also improves the robustness of the model against adversarial attacks.

[0039] During the training process, the multi-task deep learning model is trained using the enhanced annotated dataset. For each training batch, the weights of each task are adjusted based on the dynamic task weight allocation strategy, and the total loss is calculated using the uncertainty-based multi-task loss fusion method. For example, in a certain training batch, if the loss change rate of the behavior analysis task is large, indicating that it is more difficult to learn, its weight will increase accordingly. At the same time, if the uncertainty parameter of the environment detection task is small, its loss will account for a relatively larger proportion of the total loss.

[0040] Finally, the parameters of the multi-task deep learning model are optimized through the back-propagation algorithm. In each back-propagation, not only the network weights are updated, but also the uncertainty parameters and task weights. This process continues until the model converges or reaches the preset training rounds.

[0041] The multi-task learning framework adopted in this paper can handle multiple related tasks such as face recognition, behavior analysis and environmental detection at the same time, improving the overall performance and computational efficiency of the model. Secondly, the improved EfficientNet-B4 backbone network combined with dynamic convolution and SE modules enhances the feature extraction capability and adapts to input scenarios of different complexities. Thirdly, the dynamic task weight allocation strategy and the uncertainty-based loss fusion method achieve an adaptive balance between multiple tasks, avoiding the tedious process of manual parameter adjustment. Finally, the diversified data enhancement strategies, especially the introduction of adversarial samples, significantly improve the generalization ability and robustness of the model.

[0042] In an optional embodiment, The face recognition module uses the improved RetinaFace algorithm to perform face detection and uses the SE-ResNet-based age estimation model to estimate age. The steps of judging suspected minors by age threshold include: Based on a stereoscopic image dataset, face detection is performed using an improved RetinaFace algorithm, wherein the improved RetinaFace algorithm includes a channel attention module, a multi-scale context module and an adaptive sample selection strategy in a feature pyramid network, wherein the channel attention module enhances the perception of faces of different scales through adaptive average pooling and maximum pooling operations, the multi-scale context module captures multi-scale context information through different convolution kernel sizes, and the adaptive sample selection strategy dynamically adjusts the ratio of positive and negative samples according to the category probability; Inputting the detected face image into an age estimation model based on SE-ResNet, wherein the SE-ResNet model includes a Squeeze-and-Excitation module; Based on the output of the SE-ResNet age estimation model, a soft threshold strategy is used to judge minors, wherein the soft threshold strategy converts the age prediction value into the probability of a minor through a Sigmoid function, makes a final judgment based on the set probability threshold, and outputs the minor identification result.

[0043] For example, the implementation of the face recognition module involves multiple key technical links, including face detection, age estimation and minor identification. The organic combination of these links constitutes a complete minor identification system.

[0044] First, the face detection stage uses the improved RetinaFace (retinal face detection network) algorithm. As an efficient face detection method, the RetinaFace algorithm has been optimized in many aspects in this study. The improved RetinaFace algorithm introduces a channel attention module, a multi-scale context module, and an adaptive sample selection strategy in the feature pyramid network, which significantly improves the detection performance of the algorithm.

[0045] The introduction of the channel attention module aims to enhance the network's perception of faces of different scales. This module effectively captures the global information in the feature map through adaptive average pooling and maximum pooling operations. Specifically, for the input feature map, global average pooling and global maximum pooling are first performed to obtain two one-dimensional vectors. After these two vectors pass through a shared fully connected layer, they are fused by addition and processed by an activation function to finally obtain the channel attention weights. These weights are applied to the original feature map to emphasize important channels and suppress irrelevant channels. For example, for a 256-channel feature map, the channel attention module may give higher weights (such as 0.9) to channels representing facial contours and lower weights (such as 0.1) to background-related channels.

[0046] The multi-scale context module is designed to capture multi-scale context information. The module processes feature maps in parallel using convolution kernels of different sizes and then fuses the results. For example, 3x3, 5x5, and 7x7 convolution kernels can be used simultaneously to capture local, medium, and large-scale context information, respectively. This design enables the network to process faces of different sizes simultaneously, improving the robustness of detection. In practical applications, this multi-scale processing can effectively cope with face detection tasks in complex scenarios, such as detecting large-sized faces at close range and small-sized faces at a distance simultaneously.

[0047] The adaptive sample selection strategy is designed to solve the problem of imbalance between positive and negative samples during training. This strategy dynamically adjusts the ratio of positive and negative samples according to the category probability. Specifically, during training, the system calculates the probability of each sample being classified as a face. Samples with probabilities close to the decision boundary (such as probabilities between 0.4 and 0.6) are given higher weights because these samples have a greater impact on the decision boundary of the model. At the same time, the system dynamically adjusts the ratio of positive and negative samples to keep it within a reasonable range, such as 1:3. This strategy ensures that the model can fully learn the characteristics of difficult samples and improves the accuracy of detection.

[0048] After face detection is completed, the next step is age estimation. This study adopts an age estimation model based on SE-ResNet (Squeeze-Excitation Residual Network). The SE-ResNet model is an improved version of the traditional ResNet that introduces the Squeeze-and-Excitation (SE) module. The core idea of ​​the SE module is to adaptively adjust the importance of feature channels through learning. In the age estimation task, the SE module can help the model better focus on feature channels related to age and suppress irrelevant information.

[0049] The workflow of the SE module is as follows: First, the input feature map is globally averaged pooled to obtain a global description of each channel. Then, the nonlinear relationship between channels is learned through a bottleneck structure consisting of two fully connected layers. Finally, the obtained weights are normalized by the sigmoid function and multiplied with the original feature map to achieve feature recalibration. For example, for feature channels that are highly related to age, such as facial wrinkles and skin color, the SE module may assign higher weights (such as 0.9), while for less relevant feature channels such as hairstyles and accessories, lower weights (such as 0.2) may be assigned.

[0050] The output of the age estimation model is a continuous age value. In order to determine whether a person is a minor, this study adopts a soft threshold strategy. This strategy is different from the traditional hard threshold judgment (such as simply judging whether the person is under 18 years old). Instead, it converts the age prediction value into the probability of being a minor through the Sigmoid function. The characteristic of the Sigmoid function is that it can map any real number to the (0,1) interval, which makes the judgment result more flexible and robust.

[0051] Specifically, a Sigmoid function centered on 18 years old can be designed. For example, when the predicted age is 15, the Sigmoid function may give a probability of 0.9 for minors; when the predicted age is 20, the probability of 0.1 for minors may be obtained. Then, according to the needs of the actual application scenario, set an appropriate probability threshold, such as 0.7. If the probability of a minor exceeds this threshold, the system determines that the person is a minor. This soft threshold strategy takes into account the uncertainty of age estimation and can better handle the boundary situation where the age prediction value is close to 18 years old.

[0052] This paper adopts the improved RetinaFace algorithm to significantly improve the accuracy and robustness of face detection by introducing channel attention module, multi-scale context module and adaptive sample selection strategy, especially in the detection of complex scenes and faces of different scales. Secondly, the age estimation model based on SE-ResNet utilizes the characteristics of SE module to enhance the perception of age-related features and improve the accuracy of age estimation. Finally, the application of soft threshold strategy makes the judgment of minors more flexible and reliable, and can better deal with the uncertainty in age estimation.

[0053] In an optional embodiment, The behavior analysis module, based on the behavior recognition method of the improved spatiotemporal graph convolutional network, determines whether there is a ticket purchase behavior of underage students, including the following steps: Dynamically generate the connection relationship between the human body feature points in the stereoscopic image data set using an adaptive graph construction module to form an adaptive graph structure; wherein the adaptive graph construction module includes an edge predictor, and the edge predictor calculates the edge weight according to the input feature through a multi-layer neural network; Inputting the adaptive graph structure into a multi-scale spatiotemporal convolution module, wherein the multi-scale spatiotemporal convolution module comprises a graph convolution layer and a temporal convolution layer with different convolution kernel sizes; Applying a spatiotemporal attention mechanism on the output features of the multi-scale spatiotemporal convolution module, including a spatial attention submodule and a temporal attention submodule; The weighted features of the spatial attention submodule and the temporal attention submodule are input into the fully connected layer for feature fusion and dimensionality reduction; Perform behavior classification based on the fused features and output the probability distribution of behavior categories; According to the predefined behavior category mapping table, the identified behavior category probability distribution is matched with the ticket purchasing behavior of young students, and the behavior recognition result is output.

[0054] For example, the behavior analysis module performs behavior recognition based on the improved spatiotemporal graph convolutional network to determine whether there is a ticket-buying behavior of young students. This method makes full use of the spatiotemporal information of human posture and can effectively capture complex human motion features.

[0055] First, the adaptive graph construction module dynamically generates the connection relationship between the human feature points in the stereo image dataset to form an adaptive graph structure. The key to this step is the edge predictor, which calculates the edge weights based on the input features through a multi-layer neural network. Specifically, the edge predictor can be a neural network composed of multiple fully connected layers. For example, a three-layer structure can be used: the input layer receives the feature vector, the middle hidden layer uses the ReLU activation function for nonlinear transformation, and the output layer produces a vector representing the edge weight.

[0056] Assuming that the number of human feature points is 17 (which is the common number of human key points), the input of the edge predictor may be a 512-dimensional feature vector (representing the 32-dimensional features of each feature point). After being processed by the neural network, the output may be a 289-dimensional vector (17 * 17, representing all possible point pair connections). Each element in this output vector represents the weight of the connection between two feature points, and the value range is between 0 and 1. For example, if the value of the 20th element in the output vector is 0.8, this may indicate that there is a strong connection between the left shoulder and the right shoulder.

[0057] Next, the adaptive graph structure is input into the multi-scale spatiotemporal convolution module. This module contains graph convolution layers and temporal convolution layers with different convolution kernel sizes, aiming to capture spatial and temporal features of different scales. The graph convolution layer processes spatial information, while the temporal convolution layer processes temporal information. For example, graph convolution layers with convolution kernel sizes of 3, 5, and 7 can be used to capture local, medium, and large-scale spatial features, respectively. The temporal convolution layer can use 1D convolution, and the convolution kernel size can be 3, 5, and 7 frames to capture action features of different time spans.

[0058] The spatiotemporal attention mechanism is applied on the output features of the multi-scale spatiotemporal convolution module, including the spatial attention submodule and the temporal attention submodule. The spatial attention submodule focuses on the key spatial regions at each time step, while the temporal attention submodule emphasizes the important time steps in the entire sequence.

[0059] Taking the spatial attention submodule as an example, it can be implemented by calculating the importance score of each position on the feature map. Assuming the size of the feature map is 17x64 (17 key points, 64-dimensional features per point), the spatial attention mechanism may output a 17-dimensional weight vector, where each element represents the importance of the corresponding key point. For example, when recognizing the "buy ticket" action, the key points of the hands and head may get higher weights (such as 0.9), while the weight of the key points of the feet may be lower (such as 0.2).

[0060] The temporal attention submodule focuses on the key moments in the entire action sequence. For an action sequence of 60 frames, the temporal attention mechanism may output a 60-dimensional weight vector, with each element corresponding to the importance of a time step. In the ticket purchase action, the key frames of reaching out and retracting the hand may receive a higher weight (such as 0.8), while the frame of the static state may have a lower weight (such as 0.3).

[0061] The results of spatial attention and temporal attention are weighted and fused and then input into the fully connected layer for feature fusion and dimensionality reduction. This step can compress high-dimensional features to lower dimensions, such as from 1024 dimensions to 256 dimensions, to extract the most critical information.

[0062] Finally, the behavior is classified based on the fused features, and the probability distribution of the behavior category is output. For example, if 10 behavior categories are predefined, the output may be a 10-dimensional vector, with each element representing the probability of the corresponding behavior category. Assuming that the probability corresponding to the "ticket purchase" behavior is 0.85, and the probabilities of other behaviors are all lower than 0.1, the system is likely to identify the action as a ticket purchase behavior.

[0063] According to the predefined behavior category mapping table, the identified behavior category probability distribution is matched with the ticket purchasing behavior of young students. This mapping table may contain multiple behavior patterns related to ticket purchasing by young students, such as "tiptoeing to buy tickets" and "buying tickets with the help of others". If the identified behavior is highly matched with these predefined patterns (such as a similarity greater than 0.8), the system will output a positive behavior recognition result, indicating that there may be ticket purchasing behavior of young students.

[0064] The adaptive graph construction module adopted in this paper can dynamically adjust the connection relationship between the key points of the human body, improving the model's adaptability to different postures and actions. Secondly, the multi-scale spatiotemporal convolution module effectively captures the spatial and temporal features of different scales, enhancing the model's ability to understand complex actions. The spatiotemporal attention mechanism further highlights the key spatial areas and time periods, improving the accuracy of recognition. Finally, through the predefined behavior category mapping table, the system can flexibly adapt to the ticket purchasing behavior patterns of young students in different scenarios. This comprehensive optimization method not only improves the accuracy and robustness of behavior recognition, but also has good generalization ability and can be applied to various complex practical scenarios.

[0065] In an optional embodiment, The environment detection module uses the improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in the lottery store, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items. The steps include: Inputting the stereo image dataset into an improved Mask R-CNN model, wherein the improved Mask R-CNN model includes a backbone network, a feature enhancement network, an improved region proposal network, and an improved ROI Head; Extracting multi-level features through the backbone network; Inputting the multi-level features into the feature enhancement network, the feature enhancement network includes a multi-scale feature fusion module, an attention enhancement module and a residual dense block, wherein the multi-scale feature fusion module fuses features of different levels, the attention enhancement module includes a channel and spatial attention mechanism, and the residual dense block uses the multi-level features for feature extraction; Inputting the output features of the feature enhancement network into the improved region proposal network, wherein the improved region proposal network adopts a dense anchor box generation strategy to generate candidate regions including standard anchor boxes and dense anchor boxes; Inputting the candidate region and the features enhanced by the feature enhancement network into the improved ROI Head, wherein the improved ROI Head includes an adaptive feature pooling layer; introducing a contrastive learning module after the improved ROI Head, wherein the contrastive learning module improves the ability to distinguish similar objects through a contrastive learning strategy; The improved Mask R-CNN model is trained using an improved loss function, wherein the improved loss function includes Focal Loss and IoU-balanced Loss. The trained model is inferred to obtain instance segmentation results and recognition results of illegal items, wherein the illegal items include electric bicycles and high-power electrical appliances.

[0066] For example, the environment detection module is one of the core components of the lottery store security management system, which is mainly responsible for instance segmentation and identification of illegal items in the store. This module is based on the improved Mask R-CNN algorithm, and by introducing feature enhancement network and contrastive learning strategy, it significantly improves the ability to identify illegal items and distinguish similar items.

[0067] The architecture of the improved Mask R-CNN model includes a backbone network, a feature enhancement network, an improved region proposal network, and an improved ROI Head (region of interest head). The backbone network uses ResNet-101 as the basic structure and extracts multi-level features through deep residual learning. These features include low-level edge and texture information, mid-level shape and component information, and high-level semantic information. For example, for the recognition of electric vehicles, low-level features may capture the circular edge of the wheel, mid-level features may recognize the shape of the handlebars and seats, and high-level features may understand the overall concept of "electric vehicle".

[0068] The feature enhancement network is an important innovation of this model. It includes a multi-scale feature fusion module, an attention enhancement module, and a residual dense block. The multi-scale feature fusion module adopts the idea of ​​feature pyramid network (FPN) to upsample and downsample feature maps of different resolutions, and then fuse them through element-by-element addition. In this way, semantic information of different scales can be fused while maintaining high resolution. The attention enhancement module contains channel attention and spatial attention mechanisms. Channel attention uses global average pooling and maximum pooling operations to generate channel attention weights and emphasize important feature channels. Spatial attention generates spatial attention maps through convolution operations to highlight important spatial locations. For example, for identifying high-power electrical appliances, channel attention may emphasize the channel representing the power plug, while spatial attention may focus on specific parts of the appliance, such as the outdoor unit of the air conditioner. The residual dense block adopts a dense connection structure, makes full use of multi-level features, and extracts richer feature representations through multiple nonlinear transformations.

[0069] The improved region proposal network adopts a dense anchor box generation strategy. In addition to the standard anchor boxes, it also generates a series of densely distributed small-sized anchor boxes. This strategy is particularly suitable for detecting small illegal items in lottery stores, such as portable appliances. For example, for a 1024x1024 input image, the traditional method may generate about 20,000 candidate regions, while the improved method can generate more than 100,000 candidate regions, greatly improving the detection ability of small objects.

[0070] The improved ROI Head contains an adaptive feature pooling layer that can dynamically adjust the pooling parameters according to the spatial distribution of the input features. This adaptive mechanism can better handle objects of different sizes and shapes, especially for irregularly shaped illegal objects, such as some high-power electrical appliances.

[0071] The contrastive learning module introduced after the improved ROI Head is the key to improving the model's ability to distinguish similar objects. This module first generates positive and negative sample pairs for each candidate region. The positive sample pairs can be images of the same electric vehicle from different perspectives, while the negative sample pairs can be images of an electric vehicle and an ordinary bicycle with a similar appearance. Then, an encoder network (such as ResNet-50) is used to map these samples to a 128-dimensional feature space. In this feature space, the cosine similarity between sample pairs is calculated. For positive sample pairs, the goal is to make the similarity close to 1, while for negative sample pairs, the goal is to make the similarity close to -1.

[0072] Improved loss functions are used during training, including Focal Loss and IoU-balanced Loss. Focal Loss solves the problem of class imbalance by adjusting the weights of difficult and easy samples. For example, in a lottery store scenario, illegal items (such as electric bicycles) may appear much less frequently than normal items (such as seats). IoU-balanced Loss dynamically adjusts the loss weight according to the intersection-over-union (IoU) of the predicted bounding box and the true bounding box, improving the accuracy of bounding box regression.

[0073] After the model training is completed, the newly input lottery store image is inferred. First, the features are extracted through the backbone network and feature enhancement network, and then the improved region proposal network is used to generate candidate regions. These candidate regions are processed by ROIHead and then fine-grained classified by the contrastive learning module. Finally, the instance segmentation results and recognition results of illegal items are output.

[0074] The improved Mask R-CNN algorithm adopted in this paper has many beneficial effects. First, the feature enhancement network significantly improves the model's recognition ability for complex scenes and small targets through multi-scale feature fusion, attention mechanism and residual dense connection. Secondly, the improved region proposal network adopts a dense anchor box strategy to improve the detection accuracy of targets of different sizes and shapes. Furthermore, the adaptive feature pooling layer enhances the adaptability of the model to inputs of different sizes. Finally, the improved loss function effectively solves the problems of class imbalance and positioning accuracy. These improvements enable the model to more accurately identify and segment illegal items in lottery stores, such as battery vehicles and high-power electrical appliances, and maintain high performance even in complex backgrounds and occlusions. In addition, by introducing a contrastive learning strategy, the model's ability to distinguish similar items has been significantly improved, which is particularly important for distinguishing between legal and illegal items. In general, this improved algorithm not only improves the accuracy and robustness of illegal item detection, but also has good generalization ability, can adapt to the diverse environments of different lottery stores, and provides strong technical support for the safety management of lottery sales venues.

[0075] In an optional embodiment, The steps to introduce contrastive learning strategies and improve the ability to distinguish similar objects include: Inputting the historical image data set into a neural network model including a backbone network and a projection head; extracting feature representations of the historical electrical appliance images through the backbone network; inputting the feature representations into the projection head to obtain normalized feature projections; Performing data enhancement processing on the historical image data to generate enhanced historical appliance image data, wherein the data enhancement processing includes random rotation, random cropping, color jittering, Gaussian blurring, and adding noise; inputting the enhanced historical appliance image data into the neural network model to obtain feature projections of the enhanced image; calculating a supervised contrast loss based on the normalized feature projections; calculating a self-supervised contrast loss based on the feature projections of the enhanced image; Combining the supervised contrastive loss and the self-supervised contrastive loss, we get the joint contrastive learning loss; Optimizing and training the neural network model based on the joint contrastive learning loss; Use the trained neural network model to extract features and project the stereo image dataset; Based on feature extraction and projection, the similarity between the stereo image dataset and the known electrical appliance categories is calculated, the category of the stereo image dataset is determined, and the electrical appliance recognition result is obtained.

[0076] For example, introducing a contrastive learning strategy to improve the ability to distinguish similar items is an effective method, especially in the task of electrical appliance recognition. This strategy enables the model to better distinguish similar electrical appliances by learning more discriminative feature representations.

[0077] First, the historical image dataset is input into a neural network model consisting of a backbone network and a projection head. The backbone network can select a deep convolutional neural network such as ResNet-50 or ResNet-101 to extract high-level features of the image. For example, when ResNet-50 is used as the backbone network, a 2048-dimensional feature vector can be extracted. The projection head is usually composed of one or more fully connected layers, which are used to map the feature vector to a lower-dimensional representation space. For example, a multi-layer perceptron with a hidden layer of 512 dimensions and an output layer of 128 dimensions can be used as a projection head.

[0078] After extracting the feature representation of the historical appliance image through the backbone network, these features are input into the projection head. The role of the projection head is to compress the high-dimensional features into a more compact representation space while retaining key discriminant information. After being processed by the projection head, a normalized feature projection is obtained. The normalization operation usually uses L2 norm normalization, that is, each feature vector is divided by its L2 norm so that the length of all feature vectors is 1. This can limit the feature distribution to a hypersphere, which is beneficial to the subsequent similarity calculation.

[0079] Next, the historical image data is enhanced to generate enhanced historical appliance image data. Data enhancement is an important technology that can increase the diversity of training data and improve the generalization ability of the model. The specific data enhancement processing includes: Random rotation: randomly rotate the image by a certain angle, such as ±15 degrees; Random cropping: randomly cropping an area from the original image, such as cropping an area that is 80% to 100% of the original image; Color jitter: randomly adjust the brightness, contrast, saturation and hue of the image. For example, the brightness adjustment range is ±0.2, and the contrast adjustment range is 0.8~1.2; Gaussian Blur: Applies Gaussian blur to the image, and the blur radius can be randomly selected between 0 and 2 pixels; Add noise: Add Gaussian noise or salt and pepper noise to the image. The noise intensity can be set to 0~5% of the pixel value. These enhanced historical appliance image data are input into the neural network model to obtain the feature projections of the enhanced images, which will be used for subsequent self-supervised contrastive learning.

[0080] Computing the supervised contrast loss is the next key step. The supervised contrast loss is calculated based on the normalized feature projection. The core idea is to make samples of the same category close in the feature space, while samples of different categories are far away from each other. Specifically, for each sample, we calculate its similarity with samples of the same category (through inner product) and its similarity with all other samples. Ideally, the similarity of samples of the same category should be much greater than the similarity of samples of different categories.

[0081] At the same time, we also need to calculate the self-supervised contrastive loss. The self-supervised contrastive loss is calculated based on the feature projection of the enhanced image. Its goal is to make different enhanced versions of the same image close in the feature space and far away from the enhanced versions of other images. This self-supervised learning method can help the model learn more robust and universal feature representations.

[0082] Combine the supervised contrast loss and the self-supervised contrast loss to get the joint contrast learning loss. This combination can be achieved through a simple weighted sum, for example, you can give the supervised contrast loss a weight of 0.7 and the self-supervised contrast loss a weight of 0.3. The joint loss makes full use of the information of labeled data and unlabeled data, which helps to learn more effective feature representations.

[0083] The neural network model is optimized and trained based on the joint contrastive learning loss. During the training process, the Adam optimizer can be used, the initial learning rate is set to 0.0001, and the learning rate is reduced to 0.1 times the original value every 20 training cycles. The training can be carried out for 100 training cycles, and each training cycle contains 1000 iterations. In each iteration step, 32 original images and their enhanced versions are randomly selected to form a mini-batch.

[0084] After training, the trained neural network model is used to extract features and project the stereo image dataset. This step is similar to the previous feature extraction process, but now a model optimized by contrastive learning is used to extract more discriminative features.

[0085] Finally, based on feature extraction and projection, the similarity between the stereo image dataset and the known electrical appliance categories is calculated, the category of the stereo image dataset is determined, and the electrical appliance recognition result is obtained. This step usually uses the nearest neighbor classification or soft nearest neighbor classification method. Specifically, the cosine similarity between the features of the image to be classified and the average features of each category can be calculated, and the category with the highest similarity can be selected as the classification result. In order to improve reliability, a similarity threshold can be set, such as 0.8. The classification result is given only when the highest similarity exceeds this threshold, otherwise the sample is marked as "unknown category".

[0086] The appliance recognition method based on contrastive learning adopted in this paper has many beneficial effects. First, through the combination of supervised contrastive learning and self-supervised contrastive learning, the model can use both labeled data and unlabeled data, which improves the efficiency and effect of feature learning. Secondly, the contrastive learning strategy makes the features learned by the model more discriminative, especially the ability to distinguish similar objects has been significantly improved. This is particularly important for distinguishing appliances with similar appearance but different functions, such as distinguishing between electric water heaters and air-energy water heaters. Furthermore, the application of data enhancement technology improves the generalization ability of the model, enabling it to adapt to various complex practical scenarios, such as different lighting conditions, angles and partial occlusion. Finally, this method not only improves the accuracy of appliance recognition, but also enhances the model's adaptability to new categories of appliances, providing strong technical support for the accurate identification and classification of illegal appliances in lottery stores. In general, this improved algorithm provides a more reliable and efficient technical means for the safety management and standardized operation of lottery sales venues.

[0087] In an optional embodiment, Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and the steps of weighting by considering historical violation records include: A fuzzy comprehensive evaluation model is constructed, wherein the fuzzy comprehensive evaluation model includes a target layer, a criterion layer, and an indicator layer, wherein the target layer is a violation score, the criterion layer includes age factors, behavior factors, and item factors, and the indicator layer includes a minor indicator, an adult indicator, a purchase behavior indicator, a retention behavior indicator, an interaction behavior indicator, and an illegal item indicator; a machine learning algorithm is used to determine the weight of each layer in the fuzzy comprehensive evaluation model, including using historical data to train a decision tree model, generating a feature importance matrix, calculating a feature vector, and performing cross-validation to obtain a weight vector; Design membership functions to convert the output of the multi-task deep learning model into fuzzy sets, wherein the membership functions include age membership functions, behavior membership functions, and item membership functions; based on the membership functions, convert the recognition results into fuzzy sets; construct a single factor evaluation matrix, wherein the single factor evaluation matrix includes age evaluation matrix, behavior evaluation matrix, and item evaluation matrix; The fuzzy comprehensive evaluation results are calculated using the weight vector and the single factor evaluation matrix; Combine the fuzzy comprehensive evaluation results to obtain the total evaluation matrix; Multiply the weight vector by the total judgment matrix to obtain the final judgment result; The final judgment result and the preset violation grade score are weighted and summed to obtain the initial violation score; Obtain historical violation records and calculate weighting factors based on the number of historical violations; The initial violation score is weighted using the weighting factor to obtain a final violation score; Output the final violation score as the evaluation result of the lottery store’s violation behavior.

[0088] For example, based on the execution results of the multi-task deep learning model, applying the fuzzy comprehensive evaluation method to generate violation scores and considering historical violation records for weighting is an effective method for comprehensively evaluating the violations of lottery stores. This method combines the powerful feature extraction ability of deep learning and the flexible decision-making ability of fuzzy comprehensive evaluation, and can more accurately evaluate complex violations.

[0089] First, a fuzzy comprehensive evaluation model is constructed. The model adopts a hierarchical structure, including the target layer, the criterion layer, and the indicator layer. The target layer is the final violation score, the criterion layer includes age factors, behavior factors, and item factors, and the indicator layer is further subdivided into underage indicators, adult indicators, purchase behavior indicators, retention behavior indicators, interaction behavior indicators, and illegal item indicators. This hierarchical structure can comprehensively consider all aspects that affect the violation score, making the evaluation more comprehensive and systematic.

[0090] In order to determine the weights of each level in the fuzzy comprehensive evaluation model, a machine learning algorithm is used. Specifically, historical data is used to train decision tree models, such as random forests or gradient boosting decision trees. Through these models, a feature importance matrix can be generated, which reflects the degree of influence of each indicator on the final decision. For example, it may be found that the importance of the underage indicator is 0.3, the adult indicator is 0.1, the purchase behavior indicator is 0.2, the retention behavior indicator is 0.15, the interaction behavior indicator is 0.15, and the illegal item indicator is 0.1. Then, the feature vector is calculated and cross-validated to finally obtain the weight vector. Cross-validation can use 5-fold or 10-fold cross-validation to ensure the stability and generalization ability of the weights.

[0091] Next, membership functions are designed to convert the output of the multi-task deep learning model into fuzzy sets. Membership functions include age membership function, behavior membership function, and item membership function. For example, the age membership function can be designed as follows: for those under 18 years old, the membership is 1; for those aged 18-22, the membership decreases linearly from 1 to 0; for those aged 22 and above, the membership is 0. The behavior membership function can be designed based on the duration and frequency of the behavior, such as if the purchase behavior lasts for more than 5 minutes, the membership is 1; for 2-5 minutes, the membership decreases linearly; for less than 2 minutes, the membership is 0. The item membership function can be designed according to the degree of violation of the item, such as the membership of explicitly prohibited items is 1, the membership of controversial items is 0.5, and the membership of legal items is 0.

[0092] Based on the designed membership function, the recognition results are converted into fuzzy sets. Then, single-factor evaluation matrices are constructed, including age evaluation matrix, behavior evaluation matrix and item evaluation matrix. These matrices reflect the membership distribution of each factor at different levels. For example, the age evaluation matrix may show that the membership of a case at the four levels of "high violation", "moderate violation", "mild violation" and "no violation" is 0.8, 0.2, 0, and 0 respectively.

[0093] The fuzzy comprehensive evaluation results are calculated using the weight vector and single factor evaluation matrix obtained above. This step is actually a weighted combination of the weight and the evaluation results of each factor. For example, if the weight of the age factor is 0.4, the behavior factor is 0.3, and the item factor is 0.3, then the final evaluation result will take into account the influence of these three factors.

[0094] The fuzzy comprehensive evaluation results are combined to obtain the total evaluation matrix. This matrix integrates the evaluation results of all factors and indicators and provides a basis for the final violation score.

[0095] Multiply the weight vector by the total judgment matrix to get the final judgment result. This result is a vector that reflects the membership of the current case at each violation level. For example, [0.6, 0.3, 0.1, 0] may be obtained, indicating the membership distribution of the case at the four levels of high violation, medium violation, slight violation and no violation.

[0096] The final evaluation result is weighted and summed with the preset violation level score to obtain the initial violation score. For example, if the scores corresponding to the four levels are 100, 70, 30, and 0, the initial violation score is 0.6*100+0.3*70+0.1*30+0*0 = 81 points.

[0097] Obtain historical violation records and calculate a weighting factor based on the number of historical violations. The weighting factor can be designed as an increasing function, such as 1+0.1*number of historical violations. In this way, for the first violation case, the weighting factor is 1; for a case with 5 historical violation records, the weighting factor is 1.5.

[0098] The initial violation score is weighted by the weighting factor to obtain the final violation score. For example, if the initial violation score is 81 points and the number of historical violations is 3, the weighting factor is 1.3, and the final violation score is 81*1.3 = 105.3 points.

[0099] Finally, the final violation score is output as the evaluation result of the lottery store's violation. This score can be used for subsequent decision making, such as warnings, fines, or revocation of business licenses.

[0100] The violation scoring method based on fuzzy comprehensive evaluation and weighted historical violation records adopted in this paper has many beneficial effects. First, it can comprehensively consider multiple influencing factors, including age, behavior, and items, making the evaluation results more objective and comprehensive. Secondly, by using fuzzy theory, this method can better deal with uncertainty and ambiguity in the real world, avoiding the limitations of traditional hard threshold methods. Furthermore, the introduction of machine learning algorithms to determine weights enables the evaluation model to adaptively adjust the importance of each factor based on historical data, improving the accuracy and adaptability of the model. In addition, the design of weighted historical violation records can impose more severe penalties on repeated violations, which is conducive to curbing repeated violations.

[0101] Figure 2 FIG. 1 is a schematic diagram of a system for identifying violations and anomalies of lottery stores according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to install high-resolution RGB cameras, depth cameras and thermal imaging cameras in lottery stores, and use edge computing devices to collect multi-source heterogeneous image data in real time; pre-process the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth and thermal information; The second unit is used to build a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module. The multi-task deep learning model is trained and optimized using a labeled data set; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and determines suspected minors through an age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The third unit is used to input the stereoscopic image dataset into the trained multi-task deep learning model, and simultaneously perform face recognition, behavior analysis, and environmental detection tasks; based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting; based on the violation score results, a lottery store on-site inspection plan is generated.

[0102] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0103] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lottery store violation anomaly identification method, characterized in that: include: Install high-resolution RGB cameras, depth cameras, and thermal imaging cameras in lottery stores, and use edge computing devices to collect multi-source heterogeneous image data in real time; Preprocess the collected multi-source heterogeneous image data to form a stereoscopic image dataset containing spatial, depth and thermal information; Construct a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module. The multi-task deep learning model is trained and optimized using a labeled data set; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and determines suspected minors by age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved MaskR-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The stereo image dataset is input into the trained multi-task deep learning model, and face recognition, behavior analysis and environmental detection tasks are performed simultaneously. Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting. Based on the violation score results, an on-site inspection plan for lottery stores is generated.

2. The method according to claim 1, characterized in that The steps of preprocessing the collected multi-source heterogeneous image data to form a stereoscopic image data set containing spatial, depth and thermal information include: Using edge computing devices to collect multi-source heterogeneous image data in lottery stores in real time, the multi-source heterogeneous image data including RGB image data, depth image data and thermal imaging image data; Applying an improved conditional generative adversarial network to the RGB image data for image enhancement, wherein the generator of the conditional generative adversarial network adopts a U-Net structure and introduces a channel attention mechanism in the jump connection, and the discriminator adopts a PatchGAN structure, and optimizes network parameters by combining L1 loss, adversarial loss and perceptual loss function; The enhanced RGB image data, depth image data and thermal imaging image data are registered using an end-to-end registration framework based on deep learning. The end-to-end registration framework includes a feature extraction network and a transformation estimation network. The feature extraction network uses a twin structure to process image data of different modalities respectively. The transformation estimation network uses a multi-layer perceptron structure to output affine transformation parameters and optimizes the registration accuracy through a multi-modal similarity metric loss function. The registered image data is input into a multi-scale feature fusion network, which adopts an improved feature pyramid network structure, introduces a cross-modal self-attention mechanism to enhance information interaction between different modalities, and dynamically adjusts the fusion weights of different modal features through an adaptive feature aggregation module; The output of the multi-scale feature fusion network is converted into a unified tensor format to generate a stereo image dataset containing spatial, depth, and thermal information.

3. The method according to claim 1, characterized in that: Construct a multi-task deep learning model, the multi-task deep learning model includes a face recognition module, a behavior analysis module and an environment detection module, and the steps of training and optimizing the multi-task deep learning model using a labeled data set include: Constructing a multi-task deep learning model, the multi-task deep learning model includes a shared feature extraction backbone network and multiple task branch networks, the shared feature extraction backbone network adopts an improved EfficientNet-B4 structure, introduces dynamic convolution and Squeeze-and-Excitation modules, and generates feature maps of multiple scales through a multi-scale feature fusion mechanism; the multiple task branch networks include a face recognition module branch network, a behavior analysis module branch network, and an environment detection module branch network; Design a dynamic task weight allocation strategy, including building a task difficulty assessment module, dynamically adjusting task weights by monitoring the rate of change of training loss for each task, using the Softmax function to map task difficulty to a predetermined interval, and using the mapping result as a scaling factor for task weights; Formulate an uncertainty-based multi-task loss fusion method, introduce a learnable uncertainty parameter for each task, divide the loss of each task by the square of the corresponding uncertainty parameter, and add the logarithm of the product of all uncertainty parameters to obtain the total loss function; Implement data augmentation strategies, including geometric transformation and color jittering of the annotated dataset, and use the fast gradient sign method to generate adversarial samples, which are then mixed with the original samples to form an enhanced annotated dataset. The multi-task deep learning model is trained using the enhanced labeled data set. During the training process, the weight of each task is adjusted based on a dynamic task weight allocation strategy, and the total loss is calculated using an uncertainty-based multi-task loss fusion method; The multi-task deep learning model parameters are optimized through the back-propagation algorithm, and the uncertainty parameters and task weights are updated simultaneously until the multi-task deep learning model converges or reaches the preset training rounds.

4. The method according to claim 1, characterized in that: The face recognition module uses the improved RetinaFace algorithm to perform face detection and uses the SE-ResNet-based age estimation model to estimate age. The steps of judging suspected minors by age threshold include: Based on a stereoscopic image dataset, face detection is performed using an improved RetinaFace algorithm, wherein the improved RetinaFace algorithm includes a channel attention module, a multi-scale context module and an adaptive sample selection strategy in a feature pyramid network, wherein the channel attention module enhances the perception of faces of different scales through adaptive average pooling and maximum pooling operations, the multi-scale context module captures multi-scale context information through different convolution kernel sizes, and the adaptive sample selection strategy dynamically adjusts the ratio of positive and negative samples according to the category probability; Inputting the detected face image into an age estimation model based on SE-ResNet, wherein the SE-ResNet model includes a Squeeze-and-Excitation module; Based on the output of the SE-ResNet age estimation model, a soft threshold strategy is used to judge minors, wherein the soft threshold strategy converts the age prediction value into the probability of a minor through a Sigmoid function, makes a final judgment based on the set probability threshold, and outputs the minor identification result.

5. The method according to claim 1, characterized in that The behavior analysis module, based on the behavior recognition method of the improved spatiotemporal graph convolutional network, determines whether there is a ticket purchase behavior of underage students, including the following steps: Dynamically generate the connection relationship between the human body feature points in the stereoscopic image data set using an adaptive graph construction module to form an adaptive graph structure; wherein the adaptive graph construction module includes an edge predictor, and the edge predictor calculates the edge weight according to the input feature through a multi-layer neural network; Inputting the adaptive graph structure into a multi-scale spatiotemporal convolution module, wherein the multi-scale spatiotemporal convolution module comprises a graph convolution layer and a temporal convolution layer with different convolution kernel sizes; Applying a spatiotemporal attention mechanism on the output features of the multi-scale spatiotemporal convolution module, including a spatial attention submodule and a temporal attention submodule; The weighted features of the spatial attention submodule and the temporal attention submodule are input into the fully connected layer for feature fusion and dimensionality reduction; Perform behavior classification based on the fused features and output the probability distribution of behavior categories; According to the predefined behavior category mapping table, the identified behavior category probability distribution is matched with the ticket purchasing behavior of young students, and the behavior recognition result is output.

6. The method according to claim 1, characterized in that The environment detection module uses the improved Mask R-CNN algorithm to segment and identify illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; The steps of contrastive learning strategy to improve the ability to distinguish similar objects include: Inputting the stereo image dataset into an improved Mask R-CNN model, wherein the improved Mask R-CNN model includes a backbone network, a feature enhancement network, an improved region proposal network, and an improved ROI Head; Extracting multi-level features through the backbone network; Inputting the multi-level features into the feature enhancement network, the feature enhancement network includes a multi-scale feature fusion module, an attention enhancement module and a residual dense block, wherein the multi-scale feature fusion module fuses features of different levels, the attention enhancement module includes a channel and spatial attention mechanism, and the residual dense block uses the multi-level features for feature extraction; Inputting the output features of the feature enhancement network into the improved region proposal network, wherein the improved region proposal network adopts a dense anchor box generation strategy to generate candidate regions including standard anchor boxes and dense anchor boxes; Inputting the candidate region and the features enhanced by the feature enhancement network into the improved ROI Head, wherein the improved ROI Head includes an adaptive feature pooling layer; introducing a contrastive learning module after the improved ROI Head, wherein the contrastive learning module improves the ability to distinguish similar objects through a contrastive learning strategy; The improved Mask R-CNN model is trained using an improved loss function, wherein the improved loss function includes Focal Loss and IoU-balanced Loss. The trained model is inferred to obtain instance segmentation results and recognition results of illegal items, wherein the illegal items include electric bicycles and high-power electrical appliances.

7. The method according to claim 1, characterized in that The steps to introduce contrastive learning strategies and improve the ability to distinguish similar objects include: Inputting the historical image data set into a neural network model including a backbone network and a projection head; extracting feature representations of the historical electrical appliance images through the backbone network; inputting the feature representations into the projection head to obtain normalized feature projections; Performing data enhancement processing on the historical image data to generate enhanced historical appliance image data, wherein the data enhancement processing includes random rotation, random cropping, color jittering, Gaussian blurring, and adding noise; inputting the enhanced historical appliance image data into the neural network model to obtain feature projections of the enhanced image; calculating a supervised contrast loss based on the normalized feature projections; calculating a self-supervised contrast loss based on the feature projections of the enhanced image; Combining the supervised contrastive loss and the self-supervised contrastive loss, we get the joint contrastive learning loss; Optimizing and training the neural network model based on the joint contrastive learning loss; Use the trained neural network model to extract features and project the stereo image dataset; Based on feature extraction and projection, the similarity between the stereo image dataset and the known electrical appliance categories is calculated, the category of the stereo image dataset is determined, and the electrical appliance recognition result is obtained.

8. The method according to claim 1, characterized in that Based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and the steps of weighting by considering historical violation records include: A fuzzy comprehensive evaluation model is constructed, wherein the fuzzy comprehensive evaluation model includes a target layer, a criterion layer, and an indicator layer, wherein the target layer is a violation score, the criterion layer includes age factors, behavior factors, and item factors, and the indicator layer includes a minor indicator, an adult indicator, a purchase behavior indicator, a retention behavior indicator, an interaction behavior indicator, and an illegal item indicator; a machine learning algorithm is used to determine the weight of each layer in the fuzzy comprehensive evaluation model, including using historical data to train a decision tree model, generating a feature importance matrix, calculating a feature vector, and performing cross-validation to obtain a weight vector; Design membership functions to convert the output of the multi-task deep learning model into fuzzy sets, wherein the membership functions include age membership functions, behavior membership functions, and item membership functions; based on the membership functions, convert the recognition results into fuzzy sets; construct a single factor evaluation matrix, wherein the single factor evaluation matrix includes age evaluation matrix, behavior evaluation matrix, and item evaluation matrix; The fuzzy comprehensive evaluation results are calculated using the weight vector and the single factor evaluation matrix; Combine the fuzzy comprehensive evaluation results to obtain the total evaluation matrix; Multiply the weight vector by the total judgment matrix to obtain the final judgment result; The final judgment result and the preset violation grade score are weighted and summed to obtain the initial violation score; Obtain historical violation records and calculate weighting factors based on the number of historical violations; The initial violation score is weighted using the weighting factor to obtain a final violation score; Output the final violation score as the evaluation result of the lottery store’s violation behavior.

9. A lottery store violation anomaly identification system for implementing the method according to any one of claims 1 to 8, characterized in that: include: The first unit is used to install high-resolution RGB cameras, depth cameras, and thermal imaging cameras in lottery stores, using edge computing devices to collect multi-source heterogeneous image data in real time; Preprocess the collected multi-source heterogeneous image data to form a stereoscopic image dataset containing spatial, depth and thermal information; The second unit is used to build a multi-task deep learning model, which includes a face recognition module, a behavior analysis module and an environment detection module. The multi-task deep learning model is trained and optimized using a labeled data set; the face recognition module uses an improved RetinaFace algorithm to perform face detection, and uses an SE-ResNet-based age estimation model to estimate age, and determines suspected minors through an age threshold; the behavior analysis module uses an improved spatiotemporal graph convolutional network behavior recognition method to determine whether there is a ticket purchase behavior of underage students; the environment detection module uses an improved Mask R-CNN algorithm to perform instance segmentation and identification of illegal items in lottery stores, and improves the ability to identify illegal items by introducing a feature enhancement network; introduces a comparative learning strategy to improve the ability to distinguish similar items; The third unit is used to input the stereoscopic image dataset into the trained multi-task deep learning model, and simultaneously perform face recognition, behavior analysis, and environmental detection tasks; based on the execution results of the multi-task deep learning model, the fuzzy comprehensive evaluation method is used to generate violation scores, and historical violation records are considered for weighting; based on the violation score results, a lottery store on-site inspection plan is generated.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 8.

Images