Marketing operation abnormal behavior identification and early warning method and device based on multi-source information fusion

Abstract

The invention discloses a marketing operation abnormal behavior identification and early warning method and device based on multi-source information fusion, and belongs to the technical field of electricity marketing operation abnormal identification. The method comprises the steps that a set of safety risk identification data set oriented to a marketing operation site is constructed by combing potential risk factors in marketing site operation personnel and the environment; based on a deep learning technology, information such as a visual image sequence of a marketing operation site is analyzed in real time, and an intelligent recognition algorithm for abnormal behaviors of operators is designed; a machine learning model which is completely trained is compressed and simplified and is deployed in lightweight portable equipment, and a marketing field operator monitoring device which integrates the functions of video monitoring, information processing, abnormity early warning and the like is developed. And real-time monitoring and accurate identification of behaviors of field operating personnel can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormality identification of power marketing operations, and in particular to a method and device for identifying and early warning abnormal behaviors of marketing operations by integrating multi-source information. Background Art

[0002] At power distribution operations, numerous live devices pose a threat to the safety of power industry workers. Accidents can result not only in economic losses but also in casualties. The safety of power distribution operations is crucial to ensuring a stable and reliable power supply. Illegal operations are the primary cause of power accidents, as evidenced by the following: First, with the ever-expanding scale of power grid construction and the corresponding increase in grid equipment, the overall structure and operational characteristics of the grid system have undergone corresponding changes. Power workers are unfamiliar with the operating specifications of new equipment, which leads to misoperation during normal operations, resulting in accidents that endanger personal safety. Second, because power operations involve many scenarios and the operating conditions are complex and changeable, workers have a weak sense of safety. Especially when the workload is heavy, workers are more likely to lose concentration, become unresponsive, and become absent-minded. Working in this state of mind is extremely prone to safety accidents. Third, there are many people on power construction sites, and there are deficiencies in management. In particular, the work permit system, as the basic guarantee of power safety management, is sometimes viewed as cumbersome by workers, and its importance is overlooked. This leads to the failure to process work permits according to regulations in small operations, and they act solely based on experience, which creates serious safety hazards. To avoid power accidents, we should first stop illegal and irregular operations during power operations.

[0003] In traditional power plant operations, detecting and preventing operator violations, ensuring personal safety, and protecting electrical equipment typically involves standardized training to ensure strong professionalism. Furthermore, manual supervision and surveillance cameras are crucial for mitigating safety hazards. Regarding manual supervision, project managers are required to provide detailed guidance and oversight at the work site. However, given the densely populated and complex environment at the work site, it can be difficult for managers to comprehensively manage all tasks and maintain high concentration throughout the entire work phase. This makes oversight prone to errors, potentially leading to accidents. Furthermore, manual monitoring is highly subjective, making it prone to missed detections and false positives. Video surveillance primarily involves two methods: one involves manual real-time review of surveillance video. Once an emergency is detected, on-duty personnel can immediately detect and report it, quickly implementing appropriate measures to address or resolve the situation, thereby preventing potential safety hazards. The other method involves simply recording and storing all events within the surveillance area, but without immediate action. After an incident occurs, surveillance video from the corresponding time period is reviewed to provide evidence for the accident's nature. Both video surveillance methods are highly dependent on manpower, are inefficient, and lack real-time performance, and are unable to identify and warn in a timely manner before an accident occurs. Summary of the Invention

[0004] In view of this, the present invention addresses the deficiencies in the prior art and provides a method and device for identifying and warning abnormal marketing behaviors by fusion of multi-source information.

[0005] To solve the above technical problems, the present invention adopts a technical solution: a multi-source information fusion marketing operation abnormal behavior identification and early warning method, comprising: S1. Construct a hierarchical indicator system for the risk of unsafe behaviors in marketing operations and obtain the weight ranking results of unsafe behaviors; S2. Collect marketing personnel, equipment, and environmental data, perform data preprocessing, and construct a dataset of abnormal marketing behavior. S3. Based on the hierarchical indicator system for the risk of unsafe behaviors in marketing operations, perform multi-source heterogeneous data processing on the abnormal marketing operation behavior dataset and label target objects related to abnormal behaviors. S4. Use the improved YOLO model to detect the target objects marked in the marketing operation abnormal behavior dataset and perform training to obtain a marketing operation abnormal behavior recognition model; S5. Compress the marketing operation abnormal behavior identification model and deploy it in the marketing operation monitoring device to identify and issue warnings on abnormal marketing operation behaviors in real time.

[0006] Furthermore, in S1, in the risk level index system of unsafe behaviors in marketing operations, unsafe behaviors in marketing operations include not wearing insulating gloves, not wearing insulating clothing, not using insulating shielding, not wearing safety ropes, not wearing safety helmets, no dedicated supervisors at the work site, not using insulating ropes when passing supplies up and down, the overlapping part between insulating shields is less than 150mm, the insulating shields are not fixed with clamps, and there are residues on the poles at the end of the work; the weights of unsafe behaviors include frequency of occurrence, degree of danger and probability of missed detection.

[0007] Furthermore, in S2, a collection method combining real live operation scenarios with simulated non-live operation scenarios is adopted to collect marketing operation personnel, equipment, and environmental data; data preprocessing includes data enhancement, background processing, and data noise reduction.

[0008] Furthermore, in S3, multi-source heterogeneous data processing includes filtering and anomaly detection, labeling classification and feature extraction, feature-based and histogram alignment, geometric transformation and mixed noise.

[0009] Furthermore, in S4, the improvements to the YOLO model include: introducing an attention mechanism to enhance the recognition of model scenarios; introducing pre-trained models and small-sample learning technology, first loading the pre-trained YOLO model weights, and then fine-tuning them on the power operation dataset to adapt to specific application scenarios. In terms of small-sample learning, data augmentation technology is used to generate more training samples to expand the dataset, and generative adversarial networks or image translation technology are used to generate new training samples to enhance the model's adaptability to different scenarios. Meta-learning methods are used to quickly adapt to new tasks with small samples.

[0010] Furthermore, in S3, abnormal behaviors include non-sequential abnormal behaviors, sequential abnormal behaviors, and abnormal behaviors in text recognition; non-sequential abnormal behaviors include: failure to dress in accordance with prescribed specifications and failure to wear insulating gloves when testing electricity; sequential abnormal behaviors include: directly throwing materials downwards during high-altitude operations, ground personnel directly throwing materials upwards, and unmanned escalator operations; text recognition behaviors include abnormal signatures on work tickets; for non-sequential abnormal behaviors, the single-frame target detection method is used to monitor the occurrence of abnormal events; for sequential abnormal behaviors, the relevant target objects appearing in continuous video frames are first identified, and the posture evaluation algorithm is used to obtain the skeleton points of all people in the video frames. The target object coordinates and skeleton point coordinates are stored in the data middleware according to the video frame sequence, and then the abnormal task discrimination module is enabled to perform abnormal judgment; for abnormal behavior in text recognition, the OCR recognition algorithm is used to locate keywords, and then it is judged whether there is any abnormality in the signature area.

[0011] Furthermore, in S5, model compression methods include pruning, quantization, distillation, and low-rank decomposition.

[0012] The present invention also provides a multi-source information fusion marketing operation abnormal behavior identification and early warning device, including: an information collection unit, an information processing unit, an information display unit and a hardware control unit. The information collection unit includes a camera, a sensor and a communication module, which is used to collect marketing operation personnel, equipment, and environmental data, and transmit them to the information processing unit; the information processing unit includes a server, which runs system software and algorithms, and uses a marketing operation abnormal behavior identification model to analyze and process the image information and sensor information input by the information collection unit, to realize the detection of staff, work clothes, safety helmets, gloves, and whether to enter a dangerous area. If abnormal behavior is found on site, a voice alarm is issued; the information display unit includes a display, which is connected to the server, and presents the processing results of the image information and sensor information by the information processing unit in real time in the form of images.

[0013] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a safety risk identification dataset for marketing operations by analyzing potential hazards within field workers and their environments. Using deep learning technology, it analyzes visual image sequences and other information from marketing operations in real time to design an intelligent algorithm for identifying abnormal worker behavior. The fully trained machine learning model is then compressed and streamlined, deployed into a lightweight, portable device, resulting in a monitoring device for marketing operations that integrates video surveillance, information processing, and anomaly warnings. This device enables real-time monitoring and accurate identification of on-site worker behavior. Traditional safety monitoring methods often rely on manual observation and post-analysis. This technology, through an automated monitoring system, can instantly identify potential abnormal behaviors during operations, such as illegal operations, dangerous actions, or deviations from standard procedures, significantly reducing the probability of accidents. Furthermore, the intelligent warning system can predict potential risks in advance based on real-time data analysis and historical behavior patterns, providing opportunities for proactive intervention. Compared to traditional methods, this technology offers significant advantages in data processing capabilities, response speed, and identification accuracy, effectively ensuring safety and efficiency at the operations site. At the same time, through continuous self-learning and optimization, the system can continuously improve its ability to recognize complex behavioral patterns, ensuring that it maintains efficient and stable performance in an ever-changing on-site environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of the method of embodiment 1 of the present invention; Figure 2 Schematic diagram of a marketing operation abnormal behavior dataset in Example 1 of the present invention; Figure 3 This is a schematic diagram of abnormal behavior classification and identification in Example 1 of the present invention; Figure 4 This is an architectural diagram of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0016] Example 1: Figure 1 As shown, a multi-source information fusion marketing operation abnormal behavior identification and early warning method includes: S1. Construct a hierarchical indicator system for the risk of unsafe behaviors in marketing operations and obtain the weight ranking results of unsafe behaviors; S2. Collect marketing personnel, equipment, and environmental data, perform data preprocessing, and construct a dataset of abnormal marketing behavior. S3. Based on the hierarchical indicator system for the risk of unsafe behaviors in marketing operations, perform multi-source heterogeneous data processing on the abnormal marketing operation behavior dataset and label target objects related to abnormal behaviors. S4. Use the improved YOLO model to detect the target objects marked in the marketing operation abnormal behavior dataset and perform training to obtain a marketing operation abnormal behavior recognition model; S5. Compress the marketing operation abnormal behavior identification model and deploy it in the marketing operation monitoring device to identify and issue warnings on abnormal marketing operation behaviors in real time.

[0017] Specifically, in S1, in the risk level index system for unsafe behaviors in marketing operations, unsafe behaviors in marketing operations include not wearing insulating gloves, not wearing insulating clothing, not using insulating shielding, not wearing safety ropes, not wearing safety helmets, no dedicated supervisors at the work site, not using insulating ropes when passing supplies up and down, the overlap between insulating shielding is less than 150mm, insulating shielding is not fixed with clamps, and there are objects left on the pole at the end of the work; the weights of unsafe behaviors include frequency of occurrence, degree of danger, and probability of missed detection, as shown in the following table:

[0018] In S2, a combination of real live working scenes and simulated non-live working scenes is used to collect marketing operation personnel, equipment, and environmental data; data preprocessing includes data cleaning, data structuring, data registration, and data enhancement. Figure 2 shown.

[0019] In S3, multi-source heterogeneous data processing includes filtering and anomaly detection, labeling and classification, feature extraction, feature-based and histogram registration, geometric transformation, and mixed noise.

[0020] In S4, improvements to the YOLO model include: introducing an attention mechanism to enhance the recognition of model scenarios; introducing pre-trained models and small-sample learning technology, first loading the pre-trained YOLO model weights, and then fine-tuning them on the power operation dataset to adapt to specific application scenarios. In terms of small-sample learning, data augmentation technology is used to generate more training samples to expand the dataset, and generative adversarial networks or image translation technology are used to generate new training samples to enhance the model's adaptability to different scenarios. Meta-learning methods are used to quickly adapt to new tasks with small samples.

[0021] In S3, abnormal behaviors include non-sequential abnormal behaviors, sequential abnormal behaviors and abnormal behaviors of text recognition; non-sequential abnormal behaviors include: not dressing according to the prescribed specifications, not wearing insulating gloves when testing electricity; sequential abnormal behaviors include: directly throwing materials downwards during high-altitude operations, ground personnel directly throwing materials upwards and unmanned escalator operations; text recognition behaviors include abnormal signatures on work tickets; for non-sequential abnormal behaviors, the single-frame target detection method is used to monitor the occurrence of abnormal events; for sequential abnormal behaviors, the relevant target objects appearing in continuous video frames are first identified, and the posture evaluation algorithm is used to obtain the skeleton points of all people in the video frames. The target object coordinates and skeleton point coordinates are stored in the data middleware according to the video frame sequence, and then the abnormal task discrimination module is enabled to perform abnormal judgment; for abnormal behavior of text recognition, the OCR recognition algorithm is used to locate keywords, and then it is judged whether there is an abnormality in the signature area, such as Figure 3 shown.

[0022] In S5, model compression refers to a series of technologies and methods that reduce model complexity by reducing the number of model parameters and computational complexity while ensuring model performance. This makes deep learning models more suitable for deployment on resource-constrained devices, such as mobile devices and IoT terminals. The main methods of model compression include pruning, quantization, distillation, and low-rank decomposition. Pruning reduces model size by removing redundant or unimportant parameters; quantization reduces storage and computational costs by reducing the number of bits in model parameters; distillation achieves model compression by training a smaller student model to imitate the behavior of a larger teacher model; and low-rank decomposition reduces the number of parameters by decomposing the weight matrix into several low-rank matrices. Through model compression, the computational and storage costs of the model are effectively reduced.

[0023] Example 2: Figure 4As shown, an embodiment of the present invention provides a multi-source information fusion marketing operation abnormal behavior identification and early warning device, including: an information collection unit, an information processing unit, an information display unit and a hardware control unit. The information collection unit includes a camera, a sensor and a communication module, which is used to collect marketing operation personnel, equipment, and environmental data, and transmit them to the information processing unit; the information processing unit includes a server, which runs system software and algorithms, and uses a marketing operation abnormal behavior identification model to analyze and process the image information and sensor information input by the information collection unit, to realize the detection of staff, work clothes, safety helmets, gloves, and whether to enter a dangerous area. If abnormal behavior is found on site, a voice alarm is issued; the information display unit includes a display, which is connected to the server, and presents the processing results of the image information and sensor information by the information processing unit in real time in the form of images.

[0024] In this embodiment of the present invention, a marketing operation abnormal behavior identification model is deployed to a server. The deployment process first requires selecting an appropriate environment, such as the cloud, edge device, or local server, to meet the computing resource, latency, and data privacy requirements of different applications. The model is managed and scaled based on containerization technologies (such as Docker) and a microservices architecture, ensuring consistent operation across different environments and facilitating model updates and expansion. The model is encapsulated as an API service (such as an HTTP / REST API or gRPC service) to enable integration with other systems, transforming the model into an easily callable service. To ensure the efficiency and stability of the model in production environments, the model's performance and operational status are continuously monitored, with monitoring metrics including latency, throughput, and prediction accuracy. After model deployment, regular retraining and updates are required to address changes in data distribution and model aging. Optimization and acceleration methods, such as hardware acceleration (GPUs and TPUs), improve the model's inference speed and resource utilization efficiency, ensuring efficient and stable operation in real-world environments and providing reliable support for business decision-making and intelligent services.

[0025] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field can still modify or replace the specific implementation methods of the present invention with equivalents. Any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention are within the scope of protection of the claims of the present invention.

Claims

1. A multi-source information fusion marketing operation abnormal behavior identification and early warning method, characterized by: include: S1. Construct a hierarchical indicator system for the risk of unsafe behaviors in marketing operations and obtain the weight ranking results of unsafe behaviors; S2. Collect marketing personnel, equipment, and environmental data, perform data preprocessing, and construct a dataset of abnormal marketing behavior. S3. Based on the hierarchical indicator system for the risk of unsafe behaviors in marketing operations, perform multi-source heterogeneous data processing on the abnormal marketing operation behavior dataset and label target objects related to abnormal behaviors. S4. Use the improved YOLO model to detect the target objects marked in the marketing operation abnormal behavior dataset and perform training to obtain a marketing operation abnormal behavior recognition model; S5. Compress the marketing operation abnormal behavior identification model and deploy it in the marketing operation monitoring device to identify and issue warnings on abnormal marketing operation behaviors in real time.

2. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized by: In S1, in the risk level index system of unsafe behaviors in marketing operations, unsafe behaviors in marketing operations include not wearing insulating gloves, not wearing insulating clothing, not using insulating shielding, not wearing safety ropes, not wearing safety helmets, no dedicated supervisors at the work site, not using insulating ropes when passing supplies up and down, the overlapping part between insulating shields is less than 150mm, the insulating shields are not fixed with clamps, and there are residues on the poles at the end of the work; the weights of unsafe behaviors include frequency of occurrence, degree of danger and probability of missed detection.

3. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized by: In S2, a collection method combining real live working scenarios with simulated non-live working scenarios is used to collect data on marketing workers, equipment, and the environment; Data preprocessing includes data enhancement, background processing and data noise reduction.

4. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized by: In S3, multi-source heterogeneous data processing includes filtering and anomaly detection, labeling and classification, feature extraction, feature-based and histogram registration, geometric transformation, and mixed noise.

5. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized in that: In S4, improvements to the YOLO model include: introducing an attention mechanism to enhance the recognition of model scenarios; introducing pre-trained models and small-sample learning technology, first loading the pre-trained YOLO model weights, and then fine-tuning them on the power operation dataset to adapt to specific application scenarios. In terms of small-sample learning, data augmentation technology is used to generate more training samples to expand the dataset, and generative adversarial networks or image translation technology are used to generate new training samples to enhance the model's adaptability to different scenarios. Meta-learning methods are used to quickly adapt to new tasks with small samples.

6. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized in that: In S3, abnormal behaviors include non-temporal abnormal behaviors, temporal abnormal behaviors, and text recognition abnormal behaviors; Non-sequential abnormal behaviors include: failure to dress according to prescribed standards and failure to wear insulating gloves when testing electricity; sequential abnormal behaviors include: throwing materials directly downwards when working at heights, ground personnel throwing materials directly upwards, and unmanned escalator operations; text recognition behaviors include abnormal signatures on work tickets; for non-sequential abnormal behaviors, the single-frame target detection method is used to monitor the occurrence of abnormal events; for sequential abnormal behaviors, the relevant target objects appearing in continuous video frames are first identified, and the posture evaluation algorithm is used to obtain the skeleton points of all people in the video frames. The target object coordinates and skeleton point coordinates are stored in the data middleware according to the video frame sequence, and then the abnormal task discrimination module is enabled to perform abnormal judgment; for abnormal text recognition behavior, the OCR recognition algorithm is used to locate keywords, and then it is determined whether there is any abnormality in the signature area.

7. The method for identifying and warning abnormal marketing behaviors based on multi-source information fusion according to claim 1 is characterized in that: In S5, model compression methods include pruning, quantization, distillation, and low-rank decomposition.

8. A multi-source information fusion marketing operation abnormal behavior identification and early warning device, characterized by: include: An information collection unit, an information processing unit, an information display unit and a hardware control unit. The information collection unit includes a camera, a sensor and a communication module, which are used to collect data on marketing personnel, equipment and environment, and transmit them to the information processing unit; the information processing unit includes a server, which runs system software and algorithms, and uses a marketing operation abnormal behavior recognition model to analyze and process the image information and sensor information input by the information collection unit, to detect staff, work clothes, safety helmets, gloves, and whether they have entered a dangerous area. If abnormal behavior is found on site, a voice alarm will be issued; the information display unit includes a display, which is connected to the server, and presents the processing results of the image information and sensor information by the information processing unit in real time in the form of images.

Images