A safety warning method and system for amusement facilities based on image recognition

Abstract

The present invention belongs to the field of safety warning technology, and discloses a safety warning method and system for amusement facilities based on image recognition. The method comprises the following steps: A safety warning method for amusement facilities based on image recognition, comprising the following steps: constructing an image recognition model; collecting real-time monitoring video data of amusement facilities; using an image semantic segmentation model to perform image semantic segmentation; issuing a support frame abnormality alarm signal according to real-time accumulated displacement data; using a moving part fault detection model to perform moving part fault detection and issue a moving part fault alarm signal; using a moving object target detection model to perform moving object target detection and issue a moving object safety alarm signal; visualizing the alarm signal. The present invention solves the problems of large labor cost investment, large workload, low accuracy and poor timeliness in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of safety warning, and in particular relates to a method and system for safety warning of amusement facilities based on image recognition. Background Art

[0002] Amusement facilities are large-scale entertainment mechanical facilities that integrate steel structures, mechanisms, electrical and automatic control. They provide tourists with mechatronic facilities that experience the feeling of acceleration by utilizing the combination of changes in acceleration gradients. In order to maximize the stimulation effect of passengers' senses, amusement facilities are usually high-speed and complex electromechanical equipment. With the development of the size and scale of amusement facilities, the consequences of accidents in amusement facilities are becoming more and more serious. How to improve the safety and operational reliability of amusement facilities has become one of the research focuses in the current field.

[0003] In the existing technology, most of them adopt the method of setting up special personnel to supervise or formulating maintenance plans for regular maintenance. This method has high labor cost and heavy workload for the staff. With the popularization of surveillance cameras, the use of online monitoring has become an important measure to ensure the safety of amusement facilities. Although this method has reduced the occurrence of safety accidents of amusement facilities to a certain extent, there are still many loopholes objectively. The safe operation of amusement facilities is difficult to be effectively guaranteed, resulting in frequent accidents. It is impossible to accurately, automatically and timely issue safety warnings for abnormalities and failures of amusement facilities. Summary of the invention

[0004] In order to solve the problems of high labor cost, heavy workload, low accuracy and poor timeliness in the prior art, the present invention aims to provide an amusement facility safety warning method and system based on image recognition.

[0005] The technical solution adopted by the present invention is:

[0006] A method for early warning of amusement facility safety based on image recognition comprises the following steps:

[0007] Collect some historical image data, and use image recognition algorithms to build image semantic segmentation models, active component fault detection models, and moving object target detection models based on some historical image data;

[0008] Collecting real-time monitoring video data of the amusement facility, and performing frame interception on the real-time monitoring video data to obtain a number of real-time monitoring image data of continuous frames;

[0009] Inputting a number of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a number of real-time support frame segmentation images of continuous frames;

[0010] Acquire the real-time cumulative displacement data of the support frame according to a number of real-time support frame segmentation images of continuous frames, and issue a support frame abnormality alarm signal according to the real-time cumulative displacement data;

[0011] Input a plurality of real-time support frame segmentation images into a movable component fault detection model, perform movable component fault detection, obtain real-time fault detection results, and issue a movable component fault alarm signal according to the real-time fault detection results;

[0012] Input a number of real-time monitoring image data into a moving object target detection model, perform moving object target detection, obtain real-time target detection results, and issue a moving object safety alarm signal based on the real-time target detection results;

[0013] Visualize the support frame abnormality alarm signal, moving part failure alarm signal and moving object safety alarm signal.

[0014] Further, the historical image data includes historical support frame image data, historical moving component image data, and historical moving object image data;

[0015] Collect some historical image data, and use image recognition algorithms to build image semantic segmentation models, active component fault detection models, and moving object target detection models based on some historical image data sets, including the following steps:

[0016] Collect some historical support frame image data, and use image segmentation algorithm to optimize training based on some historical support frame image data to build an image semantic segmentation model;

[0017] Collect a number of historical moving parts image data, and use image recognition algorithms to perform optimization training based on the historical moving parts image data to build a moving parts fault detection model;

[0018] Collect several historical moving object image data of continuous frames, use multi-target detection algorithm to perform optimization training based on several historical moving object image data, and build a moving object target detection model.

[0019] Furthermore, a number of historical support frame image data are collected, and based on the historical support frame image data, an image segmentation algorithm is used to perform optimization training to construct an image semantic segmentation model, including the following steps:

[0020] Collect several historical support frame image data of different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions;

[0021] Preprocessing a number of historical support frame image data to obtain a first model training data set;

[0022] According to the first model training data set, an image segmentation algorithm is used to perform optimization training, build an image semantic segmentation model, and obtain corresponding historical support frame segmentation image data.

[0023] Furthermore, a number of historical moving component image data are collected, and based on the number of historical moving component image data, an image recognition algorithm is used to perform optimization training to construct a moving component fault detection model, including the following steps:

[0024] Collect image data of several historical moving parts under different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions;

[0025] Preprocessing a number of historical moving component image data and adding a number of historical support frame segmentation image data to obtain a second model training data set;

[0026] According to the second model training data set, an image recognition algorithm is used to perform optimization training to build a moving part fault detection model.

[0027] Furthermore, a plurality of historical moving object image data of continuous frames are collected, and based on the plurality of historical moving object image data, a multi-target detection algorithm is used to perform optimization training to construct a moving object target detection model, including the following steps:

[0028] Collecting several historical moving object image data of continuous frames at different shooting angles, different shooting areas and different lighting conditions;

[0029] Preprocessing a number of historical moving object image data to obtain a third model training data set;

[0030] According to the third model training data set, a multi-target detection algorithm is used to perform optimization training and build a moving object detection model.

[0031] Furthermore, a plurality of real-time monitoring image data are input into an image semantic segmentation model to perform image semantic segmentation to obtain a plurality of real-time support frame segmentation images of continuous frames, including the following steps:

[0032] Preprocessing a plurality of real-time monitoring image data of continuous frames to obtain a plurality of preprocessed real-time monitoring image data, and inputting the plurality of preprocessed real-time monitoring image data into an image semantic segmentation model;

[0033] Use the image semantic segmentation model to perform image semantic segmentation on the preprocessed real-time monitoring image data to obtain the corresponding real-time support frame segmentation area;

[0034] The grayscale values ​​of pixels in the preprocessed real-time monitoring image data, except for the real-time support frame segmentation area, are set to 0 to obtain the real-time monitoring segmentation image data;

[0035] According to the minimum envelope size of the real-time support frame segmentation area, the real-time monitoring segmentation image data is cropped to obtain the corresponding real-time support frame segmentation image;

[0036] All pre-processed real-time monitoring image data are traversed, and the same support frame target is tracked to obtain several real-time support frame segmentation images of continuous frames of the same support frame target.

[0037] Further, according to a plurality of real-time support frame segmentation images of continuous frames, real-time cumulative displacement data of the support frame is obtained, and according to the real-time cumulative displacement data, a support frame abnormality alarm signal is issued, including the following steps:

[0038] Extracting support frame contour data of the same support frame target in the real-time support frame segmentation images of consecutive frames, and taking the image center point of the real-time support frame segmentation image of the first frame as the displacement reference point;

[0039] According to the displacement reference point and the support frame profile data of the support frame target, the real-time displacement data of the same support frame target is obtained;

[0040] Traversing the real-time support frame segmentation images of consecutive frames, accumulating and calculating the real-time displacement data of the same support frame target, and obtaining the corresponding real-time cumulative displacement data;

[0041] If the real-time accumulated displacement data is greater than the displacement threshold, an abnormal alarm signal of the support frame is issued, otherwise, the movable component fault detection work is started.

[0042] Furthermore, a plurality of real-time support frame segmentation images are input into a movable component fault detection model to perform movable component fault detection to obtain real-time fault detection results, and a movable component fault alarm signal is issued according to the real-time fault detection results, including the following steps:

[0043] Inputting a number of real-time support frame segmentation images of consecutive frames into a moving component fault detection model;

[0044] Using the active component fault detection model, active component target detection is performed on the real-time support frame segmentation image to obtain a first prediction frame of the active component target;

[0045] Traversing a plurality of real-time support frame segmentation images of consecutive frames, tracking a first prediction frame of a same moving component target, and performing moving component fault detection on the moving component target in the first prediction frame to obtain a real-time fault detection result of the moving component target;

[0046] If the real-time fault detection result is that there is a fault, a moving component fault alarm signal is issued, otherwise, the moving object target detection work is started.

[0047] Furthermore, a plurality of real-time monitoring image data are input into a moving object target detection model to perform moving object target detection, obtain a real-time target detection result, and issue a moving object safety alarm signal according to the real-time target detection result, including the following steps:

[0048] Input a plurality of pre-processed real-time monitoring image data of continuous frames into a moving object target detection model;

[0049] Using a moving object detection model, perform object detection on the preprocessed real-time monitoring image data to obtain a second prediction frame of several object targets;

[0050] Track the second predicted frame of the same object target, and use the pyramid optical flow method to detect the object target and obtain the optical flow value of the object target;

[0051] If the optical flow value of the object target is greater than the optical flow value threshold, the corresponding object target is output as a moving object target, and the real-time target detection result is obtained, otherwise, the object target detection continues;

[0052] If the real-time target detection result shows that there is a moving object, a moving object safety alarm signal is issued, otherwise, the alarm signal visualization work is started.

[0053] An amusement facility safety warning system based on image recognition is used to implement an amusement facility safety warning method. The system includes a safety warning center and a plurality of monitoring cameras. The safety warning center is respectively connected to the plurality of monitoring cameras for communication. The plurality of monitoring cameras are arranged at a plurality of amusement facilities in a one-to-one correspondence.

[0054] The safety warning center includes an image recognition model building unit, a monitoring video processing unit, a semantic segmentation unit, a support frame anomaly detection unit, a moving part fault detection unit, a moving object safety detection unit, and an alarm signal visualization unit, which are connected in sequence;

[0055] An image recognition model building unit, used to collect a number of historical image data, and build an image semantic segmentation model, a moving part fault detection model, and a moving object target detection model based on the historical image data using an image recognition algorithm;

[0056] A monitoring video processing unit is used to perform frame interception on the real-time monitoring video data of the amusement facility collected by the monitoring camera to obtain a number of real-time monitoring image data of continuous frames;

[0057] A semantic segmentation unit is used to input a number of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a number of real-time support frame segmentation images of continuous frames;

[0058] A support frame abnormality detection unit is used to obtain real-time cumulative displacement data of the support frame based on a number of real-time support frame segmentation images of continuous frames, and issue a support frame abnormality alarm signal based on the real-time cumulative displacement data;

[0059] A movable component fault detection unit is used to input a plurality of real-time support frame segmentation images into a movable component fault detection model, perform movable component fault detection, obtain real-time fault detection results, and issue a movable component fault alarm signal according to the real-time fault detection results;

[0060] A moving object safety detection unit is used to input a number of real-time monitoring image data into a moving object target detection model, perform moving object target detection, obtain a real-time target detection result, and issue a moving object safety alarm signal according to the real-time target detection result;

[0061] The alarm signal visualization unit is used to visualize the support frame abnormality alarm signal, the movable component failure alarm signal and the moving object safety alarm signal.

[0062] The beneficial effects of the present invention are:

[0063] The present invention discloses a method and system for early warning of amusement facility safety based on image recognition. By introducing an image recognition algorithm and detecting real-time monitoring video data online, a corresponding alarm signal is output, thereby ensuring the safety and reliability of the operation of the amusement facility, reducing the probability of accidents, and being able to give timely early warnings of accidents and abnormalities of the amusement facility. An image semantic segmentation model and a movable component fault detection model are used to perform real-time and automatic detection of abnormalities and faults on a support frame, movable components, etc. of the amusement facility, thereby improving the practicability of early warning of amusement facility safety, reducing the investment in manpower costs, and alleviating the workload of staff. A moving object target detection model is used to perform moving object target detection in key monitoring areas, so that moving object targets are discovered in a timely manner and alarm signals are issued, thereby improving functionality.

[0064] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flowchart of the amusement facility safety warning method based on image recognition in the present invention.

[0066] Figure 2 It is a structural block diagram of the amusement facility safety warning system based on image recognition in the present invention. DETAILED DESCRIPTION

[0067] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0068] Embodiment 1:

[0069] like Figure 1 As shown, this embodiment provides an amusement facility safety warning method based on image recognition, comprising the following steps:

[0070] S1: Collect a number of historical image data, and use an image recognition algorithm to build an image semantic segmentation model, a moving component fault detection model, and a moving object target detection model based on the historical image data, wherein the historical image data includes historical support frame image data, historical moving component image data, and historical moving object image data, including the following steps:

[0071] S1-1: Collect some historical support frame image data, and use image segmentation algorithm to optimize training based on some historical support frame image data to build an image semantic segmentation model, including the following steps:

[0072] S1-1-1: Collect several historical support frame image data of different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions;

[0073] As a safety early warning solution for amusement facilities, the focus of video surveillance is placed on the support frames of amusement facilities. As the main supporting equipment for the weight of amusement facilities and tourists, the support frames are prone to displacement, affecting the overall mechanical structure of the amusement facilities and thus causing safety accidents. By collecting several historical support frame image data from different shooting angles, different amusement facilities, different lighting conditions, and different occlusion conditions, the diversity of training samples is improved, so that the image recognition model can quickly identify the support frame target and improve the accuracy of model prediction;

[0074] S1-1-2: preprocessing a number of historical support frame image data to obtain a first model training data set;

[0075] In this embodiment, the preprocessing of the historical support frame image data includes grayscale processing, Gaussian denoising, and size normalization, which improves the standardization of the image data, strengthens the learning effect of the model, and further improves the accuracy of the model prediction;

[0076] S1-1-3: Based on the first model training data set, use the image segmentation algorithm to perform optimization training, build an image semantic segmentation model, and obtain corresponding historical support frame segmentation image data;

[0077] The image segmentation algorithm used in this embodiment can adopt a gated shape convolutional neural network (GSCNN) algorithm, including a rule flow module, a shape flow module and a fusion module. The rule flow module adopts a classic convolutional neural network structure for processing semantic region information. The shape flow module is composed of multiple gated convolutional layers. The low-level feature map in the rule flow module is used to process the region boundary information. The outputs of the two modules are sent to a fusion module. In the fusion module, the outputs of the rule flow module and the shape flow module are combined using an atrous special pyramid pooling (ASPP) structure. ASPP helps to preserve multi-scale context information. Finally, the fusion module generates an object semantic region with a defined boundary to obtain an accurately segmented semantic segmentation image.

[0078] The YOLOv8 algorithm can also be used. It is a target detection algorithm in the image recognition algorithm that performs prediction based on the global information of the image. At the same time, the YOLOv8 algorithm can realize the target detection function, image segmentation function and image classification function in the image recognition task through different task settings. The YOLOv8 algorithm is used as the algorithm framework of the image semantic segmentation model to save cost investment and unify the framework with the subsequent active component fault detection model and moving object target detection model.

[0079] The YOLOv8 series includes 5 models. In this embodiment, the smallest and fastest model YOLOv8n is used to build an image semantic segmentation model, which improves the timeliness of the security warning solution;

[0080] S1-2: Collect some historical image data of active components, and use image recognition algorithm to optimize training based on some historical image data of active components to build an active component fault detection model, including the following steps:

[0081] S1-2-1: Collect image data of several historical moving parts under different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions;

[0082] There are several moving parts installed on the support frame or other positions of the amusement facilities. These moving parts undertake the action tasks of the amusement facilities and are prone to failure due to aging or damage. It is necessary to detect the failure of the moving parts in real time to improve the safety of the amusement facilities. By collecting image data of several moving parts under different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions, the diversity of training samples is improved, so that the image recognition model can quickly identify the failure of the moving parts and improve the accuracy of model prediction;

[0083] S1-2-2: pre-processing a number of historical moving component image data, and adding a number of historical support frame segmentation image data to obtain a second model training data set;

[0084] In this embodiment, the preprocessing of the historical active component image data includes grayscale processing, Gaussian denoising, and size normalization, which improves the standardization of the image data, strengthens the learning effect of the model, and further improves the accuracy of the model prediction;

[0085] S1-2-3: Based on the second model training data set, use the image recognition algorithm to perform optimization training and build a moving part fault detection model;

[0086] The image recognition algorithm used in this implementation is the smallest and fastest model YOLOv8n, which improves the timeliness of the safety warning scheme. The active component fault detection model is trained through the second model training data set, which can detect active component targets in the input image in real time and add corresponding prediction boxes.

[0087] S1-3: Collecting a number of historical moving object image data of continuous frames, using a multi-target detection algorithm to perform optimization training based on the several historical moving object image data, and constructing a moving object target detection model, including the following steps:

[0088] S1-3-1: Collect several historical moving object image data of consecutive frames at different shooting angles, different shooting areas and different lighting conditions;

[0089] During the operation of amusement facilities, or important control equipment of some amusement facilities, isolation areas need to be set up to prohibit moving objects, including people and animals, from entering, to prevent moving objects from intentionally or unintentionally damaging amusement facilities. By collecting image data of several moving objects from different shooting angles, different shooting areas and different lighting conditions, the diversity of training samples is improved, so that the image recognition model can quickly identify moving objects and improve the accuracy of model prediction;

[0090] S1-3-2: Preprocessing a number of historical moving object image data to obtain a third model training data set;

[0091] In this embodiment, the preprocessing of the historical moving object image data includes grayscale processing, Gaussian denoising, and size normalization, which improves the standardization of the image data, strengthens the learning effect of the model, and further improves the accuracy of the model prediction;

[0092] S1-3-3: Based on the third model training data set, use the multi-target detection algorithm to perform optimization training and build a moving object detection model;

[0093] The image recognition algorithm used in this implementation is the smallest and fastest model YOLOv8n, which improves the timeliness of the safety warning scheme. The moving object target detection model is trained by the third model training data set, which can detect the moving object target in the input image in real time and add the corresponding prediction box.

[0094] S2: collecting real-time monitoring video data of the amusement facility, and performing frame interception on the real-time monitoring video data to obtain a number of real-time monitoring image data of continuous frames;

[0095] S3: Inputting a plurality of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a plurality of real-time support frame segmentation images of continuous frames, including the following steps:

[0096] S3-1: Preprocessing a plurality of real-time monitoring image data of continuous frames to obtain a plurality of preprocessed real-time monitoring image data, and inputting the plurality of preprocessed real-time monitoring image data into the image semantic segmentation model;

[0097] In this embodiment, the preprocessing of the real-time monitoring image data, including grayscale processing and Gaussian denoising, improves the standardization of the image data and the accuracy of the model prediction;

[0098] S3-2: Use the image semantic segmentation model to perform image semantic segmentation on the preprocessed real-time monitoring image data to obtain the corresponding real-time support frame segmentation area;

[0099] S3-3: setting the grayscale values ​​of pixels in other areas of the preprocessed real-time monitoring image data except the real-time support frame segmentation area to 0 to obtain real-time monitoring segmentation image data;

[0100] The focus on the support frame target is strengthened, the interference of irrelevant image data on the subsequent model detection is reduced, and the accuracy of model prediction is improved;

[0101] S3-4: cropping the real-time monitoring segmentation image data according to the minimum envelope size of the real-time support frame segmentation area to obtain a corresponding real-time support frame segmentation image;

[0102] Reduce the amount of data and improve the efficiency and accuracy of subsequent model predictions;

[0103] S3-5: traverse all pre-processed real-time monitoring image data, and track the same support frame target to obtain a number of real-time support frame segmentation images of continuous frames of the same support frame target;

[0104] In this embodiment, a single target tracking (Simple Online and Realtime Tracking, SORT) algorithm is used to track the same support frame target in real time, thereby improving the accuracy of model prediction;

[0105] S4: acquiring the real-time cumulative displacement data of the support frame according to a plurality of real-time support frame segmentation images of continuous frames, and issuing a support frame abnormality alarm signal according to the real-time cumulative displacement data, including the following steps:

[0106] S4-1: extracting support frame contour data of the same support frame target in the real-time support frame segmentation images of consecutive frames, and taking the image center point of the real-time support frame segmentation image of the first frame as the displacement reference point;

[0107] S4-2: acquiring real-time displacement data of the same support frame target according to the displacement reference point and the support frame contour data of the support frame target;

[0108] Extract the first distance data from the support frame contour data of the current frame to the displacement reference point, and extract the second distance data from the support frame contour data of the next frame to the displacement reference point. According to the difference between the first distance data and the second distance data, the real-time displacement data of the same support frame target in the two frames of images can be obtained;

[0109] S4-3: traversing the real-time support frame segmentation images of consecutive frames, accumulating and calculating the real-time displacement data of the same support frame target, and obtaining corresponding real-time accumulated displacement data;

[0110] S4-4: If the real-time accumulated displacement data is greater than the displacement threshold, an abnormal alarm signal of the support frame is issued, otherwise, the active component fault detection work is started;

[0111] If the real-time accumulated displacement data is greater than the displacement threshold, the corresponding support frame target has a larger displacement than the displacement reference point, and the support frame as the weight support has an abnormality, and an alarm signal needs to be issued to remind the staff to carry out maintenance;

[0112] S5: inputting a plurality of real-time support frame segmentation images into a movable component fault detection model, performing movable component fault detection, obtaining real-time fault detection results, and issuing a movable component fault alarm signal according to the real-time fault detection results, including the following steps:

[0113] S5-1: inputting a plurality of real-time support frame segmentation images of consecutive frames into a movable component fault detection model;

[0114] S5-2: Using the moving component fault detection model, perform moving component target detection on the real-time support frame segmentation image to obtain a first prediction frame of the moving component target;

[0115] S5-3: traversing a plurality of real-time support frame segmentation images of consecutive frames, tracking a first prediction frame of the same moving component target, and performing moving component fault detection on the moving component target in the first prediction frame to obtain a real-time fault detection result of the moving component target;

[0116] Tracking the first prediction frame of the same moving part target improves the accuracy of model prediction. In the model training stage, a large number of moving part fault images are used for learning, so that the moving part fault detection model can identify a variety of moving part fault conditions, including loose moving part nuts, damaged moving parts, or moving part detachment.

[0117] S5-4: If the real-time fault detection result is that there is a fault, a moving component fault alarm signal is issued, otherwise, the moving object target detection work is started;

[0118] S6: Inputting a plurality of real-time monitoring image data into a moving object target detection model, performing moving object target detection, obtaining a real-time target detection result, and issuing a moving object safety alarm signal according to the real-time target detection result, including the following steps:

[0119] S6-1: inputting a plurality of pre-processed real-time monitoring image data of consecutive frames into a moving object target detection model;

[0120] S6-2: Using the moving object detection model, perform object detection on the preprocessed real-time monitoring image data to obtain a second prediction frame of several object targets;

[0121] S6-3: Track the second predicted frame of the same object target, and use the pyramid optical flow method to detect the object target and obtain the optical flow value of the object target;

[0122] The instantaneous speed of the pixel movement of a moving object in space on the observation imaging plane is a method of finding the corresponding relationship between the previous frame and the current frame by using the change of pixels in the time domain in the image sequence and the correlation between adjacent frames, so as to calculate the motion information of the object between adjacent frames. In other words, it is a projection from the spatial domain to the image plane. In layman's terms, finding the instantaneous speed and direction of each point in the image is optical flow.

[0123] The pyramid optical flow method is a well-known optical flow algorithm in the field of image recognition. It can be used to calculate the motion information of objects between adjacent frames in an image sequence. The basic idea is to use the grayscale difference between adjacent pixels in the image to represent the motion situation, and then gradually improve the accuracy of motion estimation through the pyramid hierarchical idea to achieve better optical flow calculation results.

[0124] S6-4: If the optical flow value of the object target is greater than the optical flow value threshold, the corresponding object target is output as a moving object target, and a real-time target detection result is obtained, otherwise, the object target detection is continued;

[0125] S6-5: If the real-time target detection result shows that there is a moving object, a moving object safety alarm signal is issued, otherwise, the alarm signal visualization work is started;

[0126] S7: Visualize the support frame abnormality alarm signal, the moving part failure alarm signal, and the moving object safety alarm signal;

[0127] The alarm signals are visualized to remind the staff to deal with the abnormalities, failures or violations of the corresponding amusement facilities, realizing the safety early warning deployment of amusement facilities and improving the safety and reliability of amusement facilities.

[0128] Embodiment 2:

[0129] like Figure 2 As shown, this embodiment provides an amusement facility safety warning system based on image recognition, which is used to implement an amusement facility safety warning method. The system includes a safety warning center and a plurality of monitoring cameras. The safety warning center is respectively connected to the plurality of monitoring cameras for communication. The plurality of monitoring cameras are arranged at a plurality of amusement facilities in a one-to-one correspondence.

[0130] The safety warning center includes an image recognition model building unit, a monitoring video processing unit, a semantic segmentation unit, a support frame anomaly detection unit, a moving part fault detection unit, a moving object safety detection unit, and an alarm signal visualization unit, which are connected in sequence;

[0131] An image recognition model building unit, used to collect a number of historical image data, and build an image semantic segmentation model, a moving part fault detection model, and a moving object target detection model based on the historical image data using an image recognition algorithm;

[0132] A monitoring video processing unit is used to perform frame interception on the real-time monitoring video data of the amusement facility collected by the monitoring camera to obtain a number of real-time monitoring image data of continuous frames;

[0133] A semantic segmentation unit is used to input a number of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a number of real-time support frame segmentation images of continuous frames;

[0134] A support frame abnormality detection unit is used to obtain real-time cumulative displacement data of the support frame based on a number of real-time support frame segmentation images of continuous frames, and issue a support frame abnormality alarm signal based on the real-time cumulative displacement data;

[0135] A movable component fault detection unit is used to input a plurality of real-time support frame segmentation images into a movable component fault detection model, perform movable component fault detection, obtain real-time fault detection results, and issue a movable component fault alarm signal according to the real-time fault detection results;

[0136] A moving object safety detection unit is used to input a number of real-time monitoring image data into a moving object target detection model, perform moving object target detection, obtain a real-time target detection result, and issue a moving object safety alarm signal according to the real-time target detection result;

[0137] The alarm signal visualization unit is used to visualize the support frame abnormality alarm signal, the movable component failure alarm signal and the moving object safety alarm signal.

[0138] The present invention discloses a method and system for early warning of amusement facility safety based on image recognition. By introducing an image recognition algorithm and detecting real-time monitoring video data online, a corresponding alarm signal is output, thereby ensuring the safety and reliability of the operation of the amusement facility, reducing the probability of accidents, and being able to give timely early warnings of accidents and abnormalities of the amusement facility. An image semantic segmentation model and a movable component fault detection model are used to perform real-time and automatic detection of abnormalities and faults on a support frame, movable components, etc. of the amusement facility, thereby improving the practicability of early warning of amusement facility safety, reducing the investment in manpower costs, and alleviating the workload of staff. A moving object target detection model is used to perform moving object target detection in key monitoring areas, so that moving object targets are discovered in a timely manner and alarm signals are issued, thereby improving functionality.

[0139] The present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. A safety warning method for amusement facilities based on image recognition, characterized in that: The steps include: Collect some historical image data, and use image recognition algorithm to build image semantic segmentation model, active component fault detection model and moving object target detection model according to some historical image data, including the following steps: Collect some historical support frame image data, and use image segmentation algorithm to optimize training based on some historical support frame image data to build an image semantic segmentation model, including the following steps: Collect several historical support frame image data of different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions; The historical image data includes historical support frame image data, historical moving component image data and historical moving object image data; Preprocessing a number of historical support frame image data to obtain a first model training data set; According to the first model training data set, an image segmentation algorithm is used to perform optimization training, an image semantic segmentation model is constructed, and corresponding historical support frame segmentation image data are obtained; Collect a number of historical moving parts image data, and use image recognition algorithms to perform optimization training based on the historical moving parts image data to build a moving parts fault detection model; Collect a number of historical moving object image data of continuous frames, use a multi-target detection algorithm to perform optimization training based on the number of historical moving object image data, and build a moving object target detection model; Collecting real-time monitoring video data of amusement facilities and performing frame capture on the real-time monitoring video data to obtain a number of real-time monitoring image data of continuous frames includes the following steps: Preprocessing a plurality of real-time monitoring image data of continuous frames to obtain a plurality of preprocessed real-time monitoring image data, and inputting the plurality of preprocessed real-time monitoring image data into an image semantic segmentation model; Use the image semantic segmentation model to perform image semantic segmentation on the preprocessed real-time monitoring image data to obtain the corresponding real-time support frame segmentation area; The grayscale values ​​of pixels in the preprocessed real-time monitoring image data, except for the real-time support frame segmentation area, are set to 0 to obtain the real-time monitoring segmentation image data; According to the minimum envelope size of the real-time support frame segmentation area, the real-time monitoring segmentation image data is cropped to obtain the corresponding real-time support frame segmentation image; Traversing all pre-processed real-time monitoring image data, and tracking the same support frame target, to obtain several real-time support frame segmentation images of continuous frames of the same support frame target; Inputting a number of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a number of real-time support frame segmentation images of continuous frames; According to a plurality of real-time support frame segmentation images of continuous frames, real-time cumulative displacement data of the support frame is obtained, and according to the real-time cumulative displacement data, a support frame abnormality alarm signal is issued, including the following steps: Extracting support frame contour data of the same support frame target in the real-time support frame segmentation images of consecutive frames, and taking the image center point of the real-time support frame segmentation image of the first frame as the displacement reference point; According to the displacement reference point and the support frame profile data of the support frame target, the real-time displacement data of the same support frame target is obtained; Traversing the real-time support frame segmentation images of consecutive frames, accumulating and calculating the real-time displacement data of the same support frame target, and obtaining the corresponding real-time cumulative displacement data; If the real-time accumulated displacement data is greater than the displacement threshold, an abnormal alarm signal of the support frame is issued, otherwise, the active component fault detection work is started; Input a plurality of real-time support frame segmentation images into a movable component fault detection model, perform movable component fault detection, obtain real-time fault detection results, and issue a movable component fault alarm signal according to the real-time fault detection results; Input a number of real-time monitoring image data into a moving object target detection model, perform moving object target detection, obtain real-time target detection results, and issue a moving object safety alarm signal based on the real-time target detection results; Visualize the support frame abnormality alarm signal, moving part failure alarm signal and moving object safety alarm signal.

2. The amusement facility safety warning method based on image recognition according to claim 1, characterized in that: Collecting a number of historical moving parts image data, and using an image recognition algorithm to perform optimization training based on the historical moving parts image data, and constructing a moving parts fault detection model, including the following steps: Collect image data of several historical moving parts under different shooting angles, different amusement facilities, different lighting conditions and different occlusion conditions; Preprocessing a number of historical moving component image data and adding a number of historical support frame segmentation image data to obtain a second model training data set; According to the second model training data set, an image recognition algorithm is used to perform optimization training to build a moving part fault detection model.

3. The amusement facility safety warning method based on image recognition according to claim 1, characterized in that: Collect several historical moving object image data of continuous frames, use multi-target detection algorithm to perform optimization training based on several historical moving object image data, and build a moving object target detection model, including the following steps: Collecting several historical moving object image data of continuous frames at different shooting angles, different shooting areas and different lighting conditions; Preprocessing a number of historical moving object image data to obtain a third model training data set; According to the third model training data set, a multi-target detection algorithm is used to perform optimization training and build a moving object detection model.

4. The amusement facility safety warning method based on image recognition according to claim 2, characterized in that: The method includes the following steps: inputting a plurality of real-time support frame segmentation images into a movable component fault detection model, performing movable component fault detection, obtaining a real-time fault detection result, and issuing a movable component fault alarm signal according to the real-time fault detection result: Inputting a number of real-time support frame segmentation images of consecutive frames into a moving component fault detection model; Using the active component fault detection model, active component target detection is performed on the real-time support frame segmentation image to obtain a first prediction frame of the active component target; Traversing a plurality of real-time support frame segmentation images of consecutive frames, tracking a first prediction frame of a same moving component target, and performing moving component fault detection on the moving component target in the first prediction frame to obtain a real-time fault detection result of the moving component target; If the real-time fault detection result is that there is a fault, a moving component fault alarm signal is issued, otherwise, the moving object target detection work is started.

5. The amusement facility safety warning method based on image recognition according to claim 1, characterized in that: Inputting a plurality of real-time monitoring image data into a moving object target detection model, performing moving object target detection, obtaining a real-time target detection result, and issuing a moving object safety alarm signal according to the real-time target detection result, comprises the following steps: Input a plurality of pre-processed real-time monitoring image data of continuous frames into a moving object target detection model; Using a moving object detection model, perform object detection on the preprocessed real-time monitoring image data to obtain a second prediction frame of several object targets; Track the second predicted frame of the same object target, and use the pyramid optical flow method to detect the object target and obtain the optical flow value of the object target; If the optical flow value of the object target is greater than the optical flow value threshold, the corresponding object target is output as a moving object target, and the real-time target detection result is obtained, otherwise, the object target detection continues; If the real-time target detection result shows that there is a moving object, a moving object safety alarm signal is issued, otherwise, the alarm signal visualization work is started.

6. An amusement facility safety warning system based on image recognition, used to implement the amusement facility safety warning method according to any one of claims 1 to 5, characterized in that: The system includes a safety warning center and a plurality of monitoring cameras, wherein the safety warning center is respectively connected to the plurality of monitoring cameras for communication, and the plurality of monitoring cameras are arranged at a plurality of amusement facilities in a one-to-one correspondence; The safety warning center includes an image recognition model building unit, a monitoring video processing unit, a semantic segmentation unit, a support frame anomaly detection unit, a moving part fault detection unit, a moving object safety detection unit and an alarm signal visualization unit connected in sequence; An image recognition model building unit, used to collect a number of historical image data, and build an image semantic segmentation model, a moving part fault detection model, and a moving object target detection model based on the historical image data using an image recognition algorithm; A monitoring video processing unit is used to perform frame interception on the real-time monitoring video data of the amusement facility collected by the monitoring camera to obtain a number of real-time monitoring image data of continuous frames; A semantic segmentation unit is used to input a number of real-time monitoring image data into an image semantic segmentation model to perform image semantic segmentation to obtain a number of real-time support frame segmentation images of continuous frames; A support frame abnormality detection unit is used to obtain real-time cumulative displacement data of the support frame based on a number of real-time support frame segmentation images of continuous frames, and to issue a support frame abnormality alarm signal based on the real-time cumulative displacement data; A movable component fault detection unit is used to input a plurality of real-time support frame segmentation images into a movable component fault detection model, perform movable component fault detection, obtain real-time fault detection results, and issue a movable component fault alarm signal according to the real-time fault detection results; A moving object safety detection unit is used to input a number of real-time monitoring image data into a moving object target detection model, perform moving object target detection, obtain a real-time target detection result, and issue a moving object safety alarm signal according to the real-time target detection result; The alarm signal visualization unit is used to visualize the support frame abnormality alarm signal, the movable component failure alarm signal and the moving object safety alarm signal.

Images