A special equipment intelligent inspection and analysis method and system based on image acquisition

Abstract

The present invention relates to the field of image recognition technology, and specifically to an intelligent inspection and analysis method and system for special equipment based on image acquisition. The method comprises the following steps: collecting a video of the special equipment during operation and extracting information on key points of a passenger's movements; calculating relative displacement curves, angle change curves, and passenger-equipment contact relationship curves of the key points of the movements in N consecutive frames to determine whether the passenger has completed the seat-taking action; if so, determining the corresponding equipment based on the spatial relationship between the passenger and the equipment and marking it as a to-be-identified equipment; extracting static structural features of the to-be-identified equipment, performing preliminary identification on the component status, and obtaining preliminary identification results; dividing the component into left and right regions and upper and lower regions, extracting dynamic geometric features of the component images in the left and right regions and upper and lower regions, performing secondary identification on the component status, and obtaining secondary identification results; and performing a logical joint judgment on the component status based on the preliminary identification results and the secondary identification results, and outputting a special equipment status identification result.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an intelligent inspection and analysis method and system for special equipment based on image acquisition. Background Art

[0002] With the development of computer vision and image processing technologies, image or video-based recognition and understanding technologies have been widely applied in various fields, including industrial quality inspection, traffic monitoring, and smart security. By collecting, analyzing, and performing semantic reasoning on target images, these technologies can implement intelligent perception functions such as object state recognition, structural feature extraction, and behavioral anomaly detection. In recent years, some research has attempted to apply image recognition technology to the inspection and analysis of specialized equipment, aiming to enhance the automation and operational safety of equipment operations and maintenance.

[0003] As a typical high-speed dynamic hanging structure, the seats of special equipment such as rotating flying chairs rotate and swing at high speed around the center during operation, involving complex visual scenes with multiple targets and multiple perspectives. Some current image recognition methods for the detection of special equipment such as rotating flying chairs mainly focus on structural integrity analysis or large target recognition in static scenes. During the operation of the equipment, the pressure rod is in a complex visual environment with dynamic changes, severe reflections, and frequent small-scale occlusion of targets. Traditional single visual detection strategies are difficult to achieve stable recognition. Secondly, traditional image recognition methods cannot distinguish between "recognition moments" and "non-recognition moments", resulting in the state where the pressure rod is not closed but the seat has not been taken being misjudged as an abnormality, resulting in a large number of false alarms or missed detections.

[0004] Therefore, how to automatically identify the identification time of each seat based on image acquisition technology without interfering with the normal operation of the equipment, and then identify the closing state of the safety pressure bar, the integrity of the surface structure and other key risk signs, so as to improve the intelligence, real-time and accuracy of equipment safety inspection, is an urgent problem to be solved.

[0005] Therefore, an intelligent inspection and analysis method and system for special equipment based on image acquisition is proposed. Summary of the Invention

[0006] The purpose of the present invention is to provide an intelligent inspection and analysis method and system for special equipment based on image acquisition, which can automatically identify the time when each seat should be identified without interfering with the normal operation of the equipment, and then identify the closing state of the safety pressure bar, the integrity of the surface structure and other key risk signs, thereby improving the intelligence, real-time and accuracy of equipment safety inspection.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An intelligent inspection and analysis method for special equipment based on image acquisition, comprising:

[0009] Collect videos of special equipment in operation and extract key information of passengers' movements in the video frames;

[0010] Detecting the key point information of the action, calculating the relative displacement curve, angle change curve, and passenger-equipment contact relationship curve of the key points in N consecutive frames, and determining whether the passenger has completed the seat-taking action;

[0011] If it is determined that the seating action is completed, the corresponding device is determined based on the spatial relationship between the passenger and the device, and marked as a device to be identified; static structural features of the device to be identified are extracted, and the component status of the device to be identified is preliminarily identified to obtain a preliminary identification result;

[0012] According to the symmetry center of each device to be identified, the components are divided into left and right regions and upper and lower regions; dynamic geometric features of the component images in the left and right regions and upper and lower regions are extracted, and the component status of the device to be identified is secondary identified to obtain a secondary identification result;

[0013] Combine the preliminary identification results and the secondary identification results to perform a logical joint judgment on the component status and output the special equipment status identification results.

[0014] Preferably, the process of extracting the passenger's action key point information includes:

[0015] Based on the deep learning human posture estimation algorithm, human body detection and key point positioning are performed on passengers in the video frames, and the passenger's action key points and corresponding key point coordinate information are extracted; the action key points include: shoulder key points, hip key points, knee key points and ankle key points; the key point coordinate information is filtered to obtain stable key point coordinates; the action key points and the stable key point coordinates are combined to obtain action key point information.

[0016] Preferably, the specific process of determining whether the passenger has completed the seat taking action is as follows:

[0017] Calculate the vertical displacement of the shoulder keypoint and the hip keypoint in N consecutive frames to obtain the relative displacement curve; calculate the angle change between the hip keypoint and the knee keypoint and between the knee keypoint and the ankle keypoint in N consecutive frames to obtain the angle change curve; calculate the spatial distance between the hip keypoint and the seat surface in N consecutive frames to obtain the passenger-equipment contact relationship curve;

[0018] When the displacement change amplitude in the relative displacement curve is less than the preset displacement change threshold, the angle in the angle change curve is stable within the preset device standard sitting angle range, the spatial distance in the passenger-device contact relationship curve is less than the preset contact threshold, and the duration exceeds the preset time length, it is determined that the seating action is completed.

[0019] Preferably, the specific process of the preliminary identification is:

[0020] Based on the passenger's key motion point information and the geometric dimensions of the device, a three-dimensional spatial relationship model of the person and the chair is constructed; and the spatial coordinate information of the device is determined using the three-dimensional spatial relationship model of the person and the chair.

[0021] Based on the result of determining whether the passenger has completed the seat-taking action and the spatial coordinate information of the device, it is determined whether each device is in a state where the components should be closed; the device in the state where the components should be closed is marked as a device to be identified;

[0022] In the image area of ​​the device to be identified, identifying the component contour line based on the edge contour extraction algorithm; extracting the static structural features of the device to be identified based on the component contour line, including the component curvature and the symmetrical position of the end points;

[0023] A preliminary identification is performed on the component status of the device to be identified based on the static structural features to obtain a preliminary identification result.

[0024] Preferably, the specific process of the secondary identification is:

[0025] According to the symmetry center of each device to be identified, the components are divided into left and right areas and upper and lower areas; the dynamic geometric features of the component images in the left and right areas and the upper and lower areas are extracted, including component contours, edge density, reflective brightness and texture features; symmetry indicators are calculated based on the dynamic geometric features, including component contour symmetry error, edge density consistency, reflective brightness offset and texture consistency; based on the symmetry indicators, the similarity between symmetrical areas is calculated, and the structural symmetry of the components of the device to be identified is secondary identified by comparing the similarities to obtain secondary identification results.

[0026] Preferably, the specific process of the logical joint judgment is:

[0027] A judgment logic table is established to perform a logical joint judgment in combination with the preliminary identification result and the secondary identification result; the judgment logic table is: when the special equipment is in a state where the components should be closed and the preliminary identification result and the secondary identification result are both actually closed, it is judged to be a normal state; when the special equipment is in a state where the components should be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be an abnormal state; when the special equipment is in a state where the components should not be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be a normal state.

[0028] Preferably, an intelligent inspection and analysis system for special equipment based on image acquisition includes:

[0029] The action key point extraction module is used to collect videos of special equipment in operation and extract the action key point information of passengers in the video frames;

[0030] A seating determination module is used to detect the key point information of the action, calculate the relative displacement curve, angle change curve and passenger-equipment contact relationship curve of the key points of the action in N consecutive frames, and determine whether the passenger has completed the seating action;

[0031] A component status preliminary identification module is configured to, if it is determined that the seating action is completed, determine the corresponding device based on the spatial relationship between the passenger and the device and mark it as a device to be identified; extract static structural features of the device to be identified, perform preliminary identification of the component status of the device to be identified, and obtain a preliminary identification result;

[0032] A component status secondary recognition module is used to divide the components into left and right regions and upper and lower regions based on the symmetry center of each device to be recognized; extract dynamic geometric features of the component images in the left and right regions and upper and lower regions, perform secondary recognition on the component status of the device to be recognized, and obtain secondary recognition results;

[0033] The component status joint judgment module is used to perform logical joint judgment on the component status based on the preliminary recognition result and the secondary recognition result, and output the special equipment status recognition result.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention extracts key movement points of the passenger's shoulders, hips, knees, ankles, and other parts of the body, and combines them with the relative displacement curves, angle change curves, and contact relationship curves between the passenger and the device in N consecutive frames to achieve dynamic analysis and accurate judgment of the passenger's seating action. It can effectively adapt to different body shapes, seating rhythms, and posture changes under different lighting conditions, avoid misjudgments due to temporary non-sitting postures, ensure that the component recognition process is initiated at the right time, and improve the accuracy and stability of the overall recognition process from the source.

[0036] 2. This invention combines the results of passenger seating motion analysis with the device's geometric coordinates to construct a three-dimensional spatial relationship model between the person and the device. It then performs preliminary component status recognition based on image structural features such as edge contours, curvature, and endpoint symmetry. This preliminary recognition process, based on real-world structural relationships and spatial constraints, offers enhanced interpretability and robustness against environmental interference. Even in the presence of certain degrees of surface obstruction, lighting variations, or image blur, component status recognition remains robust and fault-tolerant.

[0037] 3. The present invention's secondary recognition process is based on the inherent symmetry of the device's physical structure. By analyzing the consistency of the geometric features of components on both sides of the center of symmetry, it can maintain high detection performance even under harsh visual conditions such as weak textures, strong reflections, and low contrast. This physical constraint-based detection method has good generalization capabilities, eliminating the need to retrain models for different types of specialized equipment, significantly reducing system deployment and maintenance costs. Furthermore, the symmetry analysis method is robust to factors such as changes in ambient lighting and equipment wear and aging, and can adapt to gradual changes in the appearance of equipment in long-term operating environments, greatly enhancing the system's practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of a flow chart of an intelligent inspection and analysis method for special equipment based on image acquisition provided by an embodiment of the present invention;

[0039] Figure 2 A schematic structural diagram of an intelligent inspection and analysis system for special equipment based on image acquisition provided by an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a passenger seating determination process according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the initial identification process provided by an embodiment of the present invention;

[0042] Figure 5 A schematic diagram of the secondary identification process provided by an embodiment of the present invention;

[0043] Figure 6 A schematic diagram of the flow of logical joint judgment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] This invention proposes an intelligent inspection and analysis method and system for special equipment based on image acquisition. These methods and systems can automatically identify the closure status of each seat's safety lever, surface structural integrity, and other key risk indicators without disrupting the normal operation of the equipment, thereby improving the intelligence, real-time nature, and accuracy of equipment safety inspections. To illustrate the effectiveness of the method in this invention in improving the intelligence, real-time nature, and accuracy of equipment safety inspections, the following two examples will illustrate the effectiveness of the invention.

[0046] Example 1:

[0047] In an embodiment of the present application, the method proposed in the present invention is used to perform intelligent inspection and analysis on a rotating flying chair special equipment in a certain amusement park. Figure 1 This is a specific flow chart of the method of the present invention, which includes: collecting video of the special equipment during operation and extracting key point information of the passenger's movements; calculating the relative displacement curve, angle change curve and passenger-equipment contact relationship curve of the key points of the movements in N consecutive frames to determine whether the passenger has completed the seat-taking action; if so, determining the corresponding equipment based on the spatial relationship between the passenger and the equipment and marking it as the equipment to be identified; extracting the static structural features of the equipment to be identified, performing a preliminary identification of the component state, and obtaining a preliminary identification result; dividing the component into left and right areas and upper and lower areas, and extracting dynamic geometric features of the component images of the left and right areas and upper and lower areas, performing a secondary identification of the component state, and obtaining a secondary identification result; combining the preliminary identification result and the secondary identification result to perform a logical joint judgment on the component state, and outputting the special equipment state identification result. Figure 2 The following is a specific structural diagram of the system of the present invention. Figure 1 and Figure 2 The following content is described:

[0048] Collecting videos of special equipment in operation and extracting key action point information of passengers in the video frames; the process of extracting the key action point information of passengers includes:

[0049] Based on the deep learning human posture estimation algorithm, the passenger in the video frame is detected and key points are located, and the passenger's action key points and corresponding key point coordinate information are extracted; the action key points include: shoulder key points, hip key points, knee key points and ankle key points;

[0050] Filtering the key point coordinate information to obtain stable key point coordinates;

[0051] The action key point and the stable key point coordinates are combined to obtain action key point information.

[0052] Specifically, three high-definition cameras are placed around the rotating flying chair equipment, located in front, on the left, and on the right side of the equipment, to ensure full coverage of the visual range of all seats; the cameras continuously capture video data during the operation of the rotating flying chair at a frame rate of 30 frames per second;

[0053] A pre-trained human posture estimation neural network model is used to focus on extracting passenger motion key points and coordinate information, including shoulder key points, hip key points, knee key points, and ankle key points. The pre-trained human posture estimation neural network model includes: a data pre-processing unit, a feature extraction unit, a key point detection unit, a loss function calculation unit, a model optimization unit, and a verification and evaluation unit.

[0054] The data preprocessing unit is used to construct a training dataset suitable for the rotating flying chair scenario, including a data acquisition submodule, a data annotation submodule, and a data enhancement submodule. The data acquisition submodule installs a camera on the rotating flying chair equipment in the amusement park to collect video data at different stages, including passengers taking their seats, adjusting their sitting posture, and stabilizing their ride. The acquisition time covers operating scenarios with different lighting conditions, weather conditions, and population density. The data annotation submodule uses professional annotation tools to manually annotate the collected video frames, focusing on the precise pixel coordinates of the four key points of each passenger's shoulder, hip, knee, and ankle, while also annotating the human body's bounding box and body orientation information to provide accurate label data for subsequent supervised learning. The data enhancement submodule designs specific enhancement strategies for the special visual environment of the rotating flying chair, including random rotation transformation, brightness and contrast adjustment, noise addition, and geometric deformation. The data preprocessing unit also includes an image normalization submodule, which standardizes the pixel values ​​of the input image to a distribution with zero mean and unit variance to ensure the numerical stability of the training process.

[0055] The feature extraction unit uses a convolutional neural network based on a residual network structure as the backbone network, including multiple convolutional layers, batch normalization layers, activation function layers, and residual connection structures. The multiple convolutional layers use convolution kernels of different sizes to extract multi-scale features of the image, abstracting and combining visual information layer by layer from low-level edge texture features to high-level semantic features. The batch normalization layer performs feature standardization after each convolutional layer to accelerate training convergence and improve the generalization ability of the model. The activation function layer uses the ReLU activation function to introduce nonlinear transformations to enhance the expressive power of the model. The residual connection structure alleviates the gradient vanishing problem of deep networks through jump connections, enabling the network to train deeper layers to extract more complex feature representations. The output of the feature extraction unit is a multi-level feature map containing rich visual information from shallow to deep layers, providing a strong feature foundation for subsequent key point detection.

[0056] The key point detection unit realizes the precise positioning of the key points of the human body based on the heat map regression method, which includes a heat map generation subnetwork, a key point regression subnetwork and a coordinate decoding module; the heat map generation subnetwork converts the feature map output by the feature extraction unit into a probability heat map corresponding to each key point. The size of each heat map is proportional to the input image, and the pixel value in the heat map represents the probability of the corresponding key point at that position; for the four key points of shoulder, hip, knee and ankle, the network generates four independent heat map channels, each channel is responsible for the detection of one key point; the key point regression subnetwork gradually restores the resolution of the feature map to a size close to the input image through multi-layer convolution and upsampling operations, ensuring the accuracy of key point positioning; the upsampling process adopts a combination of transposed convolution and bilinear interpolation, which not only ensures computational efficiency but also maintains the integrity of feature information. The coordinate decoding module extracts the precise coordinates of action key points from the generated heat map. By finding the peak position in the heat map and performing sub-pixel coordinate refinement, it ultimately outputs the two-dimensional coordinates of each key point in the image coordinate system. The key point detection unit also integrates an occlusion handling mechanism. When a key point is occluded or invisible, the network can output a corresponding confidence score for reliability judgment in subsequent processing.

[0057] The loss function calculation unit designs a combination of multiple loss functions to guide model training optimization, including heat map loss calculation module, key point position loss calculation module and regularization loss calculation module; the heat map loss calculation module uses the mean square error loss function to calculate the difference between the predicted heat map and the true heat map, and optimizes the network to generate accurate key point probability distribution by minimizing this difference; the key point position loss calculation module directly calculates the Euclidean distance between the predicted key point coordinates and the true annotation coordinates, and uses the smooth L1 loss function to reduce the impact of outliers on the training process; at the same time, the key point visibility weight is introduced to reduce the weight of occluded key points in the loss calculation to avoid the negative impact of unreliable annotations on model training; the regularization loss calculation module uses weight decay technology to prevent model overfitting, and encourages the model to learn more generalized feature representations by adding the L2 norm term of the network parameters to the loss function; the overall loss function combines the various losses in a weighted summation manner, and the weight coefficient is tuned according to the characteristics of the rotating flying chair scene; the loss function calculation unit also implements a dynamic weight adjustment mechanism, which adaptively adjusts the weight ratio according to the convergence of each loss during training;

[0058] The model optimization unit is used for iterative updating of neural network parameters and control of the training process, including an optimizer configuration module, a learning rate scheduling module, and a gradient processing module. The optimizer configuration module uses an adaptive moment estimation optimizer, with the optimizer's initial learning rate set to one thousandth and the momentum parameter set to 0.9 to balance convergence speed and training stability. The learning rate scheduling module implements a multi-stage learning rate decay strategy, using a larger learning rate at the beginning of training to quickly converge to a better area, and gradually reducing the learning rate for fine-tuning as training progresses to fully learn data features and converge to the global optimal solution. The model optimization unit also integrates an early stopping mechanism and a model checkpoint saving function, which automatically stops training when the performance on the validation set no longer improves to avoid overfitting. The checkpoint saving function regularly saves the best model parameters during training to ensure that training can be resumed from the most recent checkpoint if training is unexpectedly interrupted.

[0059] Kalman filtering is performed on the extracted coordinate information of key action points to eliminate coordinate jitter caused by device rotation, lighting changes, or slight passenger movements, ensuring a stable key point coordinate sequence.

[0060] The action key points are combined with the filtered stable key point coordinate sequence to form the action key point information.

[0061] By using a deep learning human posture estimation algorithm to accurately extract information on four key areas of a passenger's body: shoulders, hips, knees, and ankles, a stable and reliable foundation for human motion recognition is established in the complex and dynamic environment of a rotating flying chair. The extraction of key motion points can effectively address challenges such as rapidly changing lighting, complex and changing backgrounds, and diverse clothing worn by people in rotating flying chair scenes. Filtering eliminates key point coordinate jitter caused by device rotation and passenger micro-movements, ensuring the accuracy of subsequent seated action judgments. Refined recognition at the key point level enables the system to understand the specific changes in a passenger's posture, providing a reliable data foundation for accurately determining whether the standard seated action has been completed, while also providing precise spatial anchors for subsequent modeling of the spatial relationship between the person and the chair.

[0062] Furthermore, the key point information of the action is detected, and the relative displacement curve, angle change curve and passenger-equipment contact relationship curve of the key point of the action in N consecutive frames are calculated to determine whether the passenger has completed the seat-taking action; Figure 3 ;

[0063] The specific process of judging whether the passenger has completed the seat-taking action is as follows:

[0064] Calculate the vertical displacement of the shoulder key points and hip key points in N consecutive frames to obtain the relative displacement curve;

[0065] Calculate the angle changes between the hip key point and the knee key point and between the knee key point and the ankle key point in N consecutive frames to obtain the angle change curve;

[0066] Calculate the spatial distance between the hip key point and the seat surface in N consecutive frames to obtain the passenger-equipment contact relationship curve;

[0067] When the displacement change amplitude in the relative displacement curve is less than the preset displacement change threshold, the angle in the angle change curve is stable within the preset device standard sitting angle range, the spatial distance in the passenger-device contact relationship curve is less than the preset contact threshold, and the duration exceeds the preset time length, it is determined that the seating action is completed.

[0068] Specifically, continuous frame detection and analysis are performed on the extracted action key point information, focusing on calculating the motion characteristic curves of the action key points in N consecutive frames (in this embodiment, N is fifteen frames, corresponding to half a second). This includes the relative displacement curve, the angle change curve, and the passenger-equipment contact relationship curve.

[0069] The vertical displacement of the shoulder and hip keypoints is calculated over fifteen consecutive frames. By analyzing the relative motion trajectories of the two keypoints, a relative displacement curve is generated, reflecting the change in the passenger's center of gravity. When a passenger transitions from standing to sitting, the hip keypoint significantly drops, while the shoulder keypoint decreases less, forming a characteristic displacement pattern.

[0070] The angle changes between the hip keypoint and the knee keypoint, and between the knee keypoint and the ankle keypoint, were calculated for fifteen consecutive frames to generate an angle change curve reflecting the degree of leg bending. In a standard sitting position on a rotating chair, the angle between the thigh and calf should remain within a specific range. Deviations from this range indicate that the passenger is not seated correctly.

[0071] The 3D distance between the hip keypoint and the corresponding seat surface is calculated for 15 consecutive frames to generate a passenger-equipment contact curve. By building a 3D model of the seat, the distance between the passenger's hips and the seat surface is accurately calculated. When the distance falls below a preset contact threshold, the passenger is confirmed to have established effective contact with the seat.

[0072] When the displacement change amplitude in the relative displacement curve is less than the preset displacement change threshold (indicating that the passenger's body is stable), the angle in the angle change curve is stable within the preset device standard sitting angle range (indicating that the leg posture is correct), and the spatial distance in the passenger-device contact relationship curve is less than the preset contact threshold (indicating that the passenger has formed effective contact with the seat), and the duration for which the above three conditions are met simultaneously exceeds the preset time length (set to 3 seconds in this embodiment), the system determines that the passenger has completed the seating action.

[0073] By analyzing the relative displacement curves, angle change curves, and contact relationship curves of key passenger points in consecutive frames, we achieve intelligent semantic understanding of passenger seating movements in the rotating chair scenario. This allows us to accurately identify the complete process of a passenger's movements from standing and adjusting to finally taking a stable seat, effectively distinguishing between true seating and temporary adjustments, and avoiding misjudgments caused by passengers tidying their clothes or adjusting their backpacks. By setting a duration threshold, we ensure that the pressure bar status detection is only triggered when the passenger is truly and stably seated. This unique design effectively addresses the fact that passengers on rotating chairs need time to adapt to the centrifugal force and find a comfortable sitting position, ensuring that the pressure bar status detection is performed at the most appropriate time, improving the accuracy and reliability of the overall detection process.

[0074] Furthermore, if it is determined that the seating action is completed, the corresponding device is determined based on the spatial relationship between the passenger and the device, and marked as the device to be identified; the static structural features of the device to be identified are extracted, and the component status of the device to be identified is preliminarily identified to obtain a preliminary identification result; and reference is made to Figure 4 The specific process of the preliminary identification is as follows:

[0075] Based on the key points of the passenger's movements and the geometric dimensions of the equipment, a three-dimensional spatial relationship model between people and equipment is constructed;

[0076] Determining the spatial coordinate information of the device using the human-device three-dimensional spatial relationship model;

[0077] Based on the result of determining whether the passenger has completed the seat-taking action and the spatial coordinate information of the device, it is determined whether each device is in a state where the components should be closed; the device in the state where the components should be closed is marked as a device to be identified;

[0078] In the image area of ​​the device to be identified, identifying the component contour line based on the edge contour extraction algorithm; extracting the static structural features of the device to be identified based on the component contour line, including the component curvature and the symmetrical position of the end points;

[0079] A preliminary identification is performed on the component status of the device to be identified based on the static structural features to obtain a preliminary identification result.

[0080] Specifically, the human-equipment three-dimensional spatial relationship model includes the static geometric information of the seat and the dynamic spatial transformation relationship during the rotation of the equipment;

[0081] The spatial coordinate information of each seat is determined using the constructed human-chair three-dimensional spatial relationship model; the X-axis of the spatial coordinate information is the horizontal direction pointing directly in front of the rotating flying chair, and the positive direction is the main viewing direction of the rotating flying chair facing the tourists; the Y-axis is the horizontal direction perpendicular to the X-axis, and the positive direction is the tangent direction of the rotating flying chair's counterclockwise rotation; the Z-axis is the vertical upward direction, with the positive direction pointing to the sky, coinciding with the rotation axis of the rotating flying chair;

[0082] Based on the results of determining whether the passenger has completed the seat-down action and combined with the seat's spatial coordinate information, the system determines whether each seat is in a state where the pressure lever should be closed. For seats where passengers are confirmed to be seated, the system marks their pressure lever status as "should be closed" and marks these seats as devices to be identified.

[0083] Within the seat image area marked as the device to be identified, the contour of the pressure rod is identified based on the pre-trained Canny edge detection algorithm. Based on the extracted contour of the pressure rod, the static structural features of the device to be identified are extracted, including the curvature features and endpoint symmetric position features of the pressure rod. The curvature features reflect the degree of bending of the pressure rod. The pressure rod in the closed state presents a complete arc curvature distribution, while the pressure rod in the open state shows an incomplete curvature pattern. The endpoint symmetric position features determine whether the pressure rod forms a closed geometric structure by analyzing the spatial position relationship between the two ends of the pressure rod.

[0084] Based on the extracted static structural features, the pressure rod state of the device to be identified is preliminarily identified; by establishing feature templates of closed pressure rods and open pressure rods, a method combining template matching and shape analysis is adopted to output the preliminary identification results of each seat to be identified, including preliminary judgment of closed and preliminary judgment of open.

[0085] By constructing a three-dimensional spatial model of the human-chair relationship and performing preliminary identification based on static structural features, the system achieved preliminary identification in the monitoring of pressure levers in rotating flying chairs. This system leverages the inherent geometric properties of the pressure levers in rotating flying chairs. By analyzing structural features such as the curvature and symmetrical position of the pressure levers' endpoints, it can accurately identify the basic state of the pressure levers in complex visual environments. This three-dimensional human-chair spatial model enables the system to precisely locate the pressure lever position for each seat. Combined with passenger motion key point detection technology, it accurately determines whether special equipment is in the "component closed state." Specifically, the system marks the corresponding device as pending for identification only when a passenger has completed the seated motion (the relative displacement curve is stable, the angle change curve conforms to the standard sitting posture, and the contact relationship curve meets the contact conditions). This fundamentally resolves the ambiguity that "a component open does not necessarily indicate an abnormality" and provides reliable assurance for accurate pressure lever location. Using an edge contour extraction algorithm to identify the pressure lever outline effectively mitigates visual interference such as metal surface reflections and lighting fluctuations in the rotating flying chair environment, laying a solid foundation for subsequent secondary recognition.

[0086] Furthermore, according to the symmetry center of each device to be identified, the components are divided into left and right regions and upper and lower regions, and dynamic geometric features of the component images of the left and right regions and upper and lower regions are extracted; based on the dynamic geometric features, the component status of the device to be identified is secondary identified to obtain a secondary identification result; referring to Figure 5 The specific process of the secondary identification is as follows:

[0087] Based on the symmetry center of each device to be identified, the component is divided into left and right areas and upper and lower areas;

[0088] Extracting dynamic geometric features of the component images in the left and right regions and the upper and lower regions, including component contours, edge density, reflective brightness, and texture features;

[0089] Calculating symmetry indicators based on the dynamic geometric features, including part profile symmetry error, edge density consistency, reflective brightness offset, and texture consistency;

[0090] The similarity between the symmetrical regions is calculated based on the symmetry index, and the structural symmetry of the components of the device to be identified is secondary identified by comparing the similarities to obtain a secondary identification result.

[0091] Specifically, for the pressure rod structure of the rotating flying chair, the left and right areas refer to the symmetrical parts of the pressure rod on both sides of the seat center line, and the upper and lower areas refer to the symmetrical parts of the pressure rod above and below the horizontal reference line.

[0092] The dynamic geometric features of the pressure bar images in the left and right areas and the upper and lower areas are extracted, including the fine features of the pressure bar outline, the density distribution of edge pixels, the reflective brightness changes of the metal surface, and the texture features of the pressure bar surface.

[0093] The symmetry error of the compression rod contour is quantified by comparing the contour shape differences between the left and right regions and the upper and lower regions. The left and right contours and the upper and lower contours of the closed compression rod should show a high degree of mirror symmetry.

[0094] Edge density consistency reflects the uniformity of edge pixel distribution within a symmetrical region. A complete strut structure should have similar edge density at symmetrical positions.

[0095] Reflection brightness deviation is detected by analyzing the spatial distribution of reflection on the metal pressure rod surface to detect whether there is abnormal brightness asymmetry;

[0096] Texture consistency compares the texture similarity of symmetrical regions through texture analysis methods such as local binary patterns.

[0097] By weighted fusion of various symmetry indicators, a comprehensive evaluation score reflecting the structural integrity of the compression rod is formed, and the similarity between symmetrical areas is obtained; when the similarity score is higher than the preset similarity threshold, the compression rod is judged to be in a normal closed state; when the similarity score is lower than the preset similarity threshold, it is judged that the compression rod may be in an abnormal situation of not being completely closed.

[0098] Through the secondary recognition method based on symmetry analysis, full use is made of the natural structural characteristics of the rotating flying chair seat and the pressure rod, which are highly symmetrical under normal circumstances. If there is an abnormality in the pressure rod on one side, the visual symmetry will be broken; by analyzing the symmetry index of dynamic geometric features, more accurate status verification can be performed based on preliminary recognition. This detection method based on physical structural laws is naturally robust to problems such as strong reflections, shadow changes, partial occlusions, etc. in the rotating flying chair environment. Compared with methods that rely solely on deep learning models, it is more stable and reliable. By calculating multi-dimensional indicators such as component contour symmetry error, edge density consistency, reflective brightness offset and texture consistency, the integrity of the pressure rod can be verified from multiple angles, significantly improving the detection accuracy under complex visual conditions and providing reliable technical guarantees for the safe operation of the rotating flying chair.

[0099] Furthermore, the component status is logically judged by combining the preliminary recognition result and the secondary recognition result, and the special equipment status recognition result is output. Figure 6 The specific process of the logical joint judgment is:

[0100] A judgment logic table is established to perform a logical joint judgment in combination with the preliminary identification result and the secondary identification result; the judgment logic table is: when the special equipment is in a state where the components should be closed and the preliminary identification result and the secondary identification result are both actually closed, it is judged to be a normal state; when the special equipment is in a state where the components should be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be an abnormal state; when the special equipment is in a state where the components should not be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be a normal state.

[0101] Specifically, when the special equipment is in a state where parts should be closed, it is confirmed that there are passengers seated; when the special equipment is in a state where parts should not be closed, it is confirmed that there are no passengers seated.

[0102] Through a logical joint judgment method, the results of the initial and secondary identifications are logically integrated, which can accurately distinguish between normal and abnormal states. In particular, it solves the problem of judging whether the opening of the hollow seat pressure rod in the rotating flying chair scene is a normal phenomenon rather than a safety hazard. The logical joint judgment method reduces the misjudgment that may occur with a single detection method through multiple verifications. When the results of the two identification methods are consistent, it provides a high-confidence judgment. When the results are inconsistent, it can trigger further analysis or manual confirmation, ensuring the reliability of the system in complex situations. This comprehensive decision-making mechanism enables the rotating flying chair's pressure rod status monitoring system to have judgment capabilities similar to those of human safety inspectors, which can minimize false alarms while ensuring safety, thereby improving practical value and operational efficiency.

[0103] This embodiment achieves a fundamental shift from passive manual inspection to active intelligent monitoring in the rotating flying chair pressure bar status monitoring scenario by constructing a complete technical process of image acquisition, motion recognition, spatial relationship modeling, dual detection and logical judgment. It can simultaneously monitor the pressure bar status of up to twenty-four seats during the high-speed rotation of the rotating flying chair, thereby improving detection efficiency and safety assurance levels. By taking the passenger's seating behavior as a prerequisite for judging the pressure bar status, it effectively solves the problem of the natural opening of the empty seat pressure bar due to centrifugal force being misjudged as abnormal during the operation of the rotating flying chair, enabling the system to accurately distinguish between the normal pressure bar opening state and the real safety hazard, significantly reducing the false alarm rate and improving the practical value of the monitoring system. It should be noted that all thresholds in this embodiment are obtained through expert experience.

[0104] Example 2:

[0105] In Example 1, the method proposed in this invention successfully automatically identified the safety lever closure status, surface structural integrity, and other key risk indicators for each seat without interfering with the normal operation of the equipment, thereby enhancing the intelligence, real-time nature, and accuracy of equipment safety inspections. To further verify the effectiveness of this invention, the present invention also conducted intelligent inspection and analysis on a special type of rotating flying chair in another large amusement park.

[0106] The action key point extraction module is used to collect videos of special equipment in operation and extract the action key point information of passengers in the video frames. The process of extracting the action key point information of passengers includes:

[0107] Based on the deep learning human posture estimation algorithm, the passenger in the video frame is detected and key points are located, and the passenger's action key points and corresponding key point coordinate information are extracted; the action key points include: shoulder key points, hip key points, knee key points and ankle key points;

[0108] Filtering the key point coordinate information to obtain stable key point coordinates;

[0109] The action key point and the stable key point coordinates are combined to obtain action key point information.

[0110] Furthermore, a seating determination module is configured to detect the key point information of the action, calculate the relative displacement curve, angle change curve, and passenger-device contact relationship curve of the key points of the action in N consecutive frames, and determine whether the passenger has completed the seating action;

[0111] The specific process of judging whether the passenger has completed the seat-taking action is as follows:

[0112] Calculate the vertical displacement of the shoulder key points and hip key points in N consecutive frames to obtain the relative displacement curve;

[0113] Calculate the angle changes between the hip key point and the knee key point and between the knee key point and the ankle key point in N consecutive frames to obtain the angle change curve;

[0114] Calculate the spatial distance between the hip key point and the seat surface in N consecutive frames to obtain the passenger-equipment contact relationship curve;

[0115] When the displacement change amplitude in the relative displacement curve is less than the preset displacement change threshold, the angle in the angle change curve is stable within the preset device standard sitting angle range, the spatial distance in the passenger-device contact relationship curve is less than the preset contact threshold, and the duration exceeds the preset time length, it is determined that the seating action is completed.

[0116] First, there is a key issue in the operation of special equipment: traditional image recognition methods are unable to distinguish between "recognition moments" and "non-recognition moments", resulting in the state where the pressure lever is not closed but the passenger has not yet sat down being misjudged as an abnormality, resulting in a large number of false alarms or missed detections. To avoid this problem, the present invention starts from passenger behavior and considers "whether the status of equipment components can be detected only after the passenger has actually sat down, thereby establishing a causal relationship between recognition and usage timing." Therefore, the present invention proposes a method to drive the device recognition trigger point based on the semantics of the passenger's action as a leading condition, establishes a logical chain of "action completion → device recognition that should be closed → state recognition", eliminates the problem of false recognition caused by timing misalignment, and realizes intelligent perception of "recognition trigger nodes" in special equipment operation scenarios. By extracting key points such as the passenger's knees, ankles, and hips frame by frame, multiple time series such as displacement curves, angle change curves, and human-chair contact curves are constructed, and whether the passenger has completed the sitting action is determined based on a triple threshold. This action judgment logic forms a logical linkage with the equipment closure state, significantly improving the accuracy of the recognition logic.

[0117] Furthermore, the component status preliminary identification module is configured to, if it is determined that the seating action is completed, determine the corresponding device based on the spatial relationship between the passenger and the device, and mark it as a device to be identified; extract the static structural features of the device to be identified, perform preliminary identification of the component status of the device to be identified, and obtain a preliminary identification result; the specific process of the preliminary identification is as follows:

[0118] Based on the key points of the passenger's movements and the geometric dimensions of the equipment, a three-dimensional spatial relationship model between people and equipment is constructed;

[0119] Determining the spatial coordinate information of the device using the human-device three-dimensional spatial relationship model;

[0120] Based on the result of determining whether the passenger has completed the seat-taking action and the spatial coordinate information of the device, it is determined whether each device is in a state where the components should be closed; the device in the state where the components should be closed is marked as a device to be identified;

[0121] In the image area of ​​the device to be identified, identifying the component contour line based on the edge contour extraction algorithm; extracting the static structural features of the device to be identified based on the component contour line, including the component curvature and the symmetrical position of the end points;

[0122] A preliminary identification is performed on the component status of the device to be identified based on the static structural features to obtain a preliminary identification result.

[0123] Furthermore, a component status secondary recognition module is used to divide the component into left and right regions and upper and lower regions according to the symmetry center of each device to be recognized, and extract dynamic geometric features of the component images in the left and right regions and upper and lower regions; perform secondary recognition of the component status of the device to be recognized based on the dynamic geometric features to obtain a secondary recognition result; the specific process of the secondary recognition is as follows:

[0124] Based on the symmetry center of each device to be identified, the component is divided into left and right areas and upper and lower areas;

[0125] Extracting dynamic geometric features of the component images in the left and right regions and the upper and lower regions, including component contours, edge density, reflective brightness, and texture features;

[0126] Calculating symmetry indicators based on the dynamic geometric features, including part profile symmetry error, edge density consistency, reflective brightness offset, and texture consistency;

[0127] The similarity between the symmetrical regions is calculated based on the symmetry index, and the structural symmetry of the components of the device to be identified is secondary identified by comparing the similarities to obtain a secondary identification result.

[0128] The lever designs of equipment such as rotating chairs and roller coasters generally present left-right symmetry. Abnormal lever states often destroy the original left-right or top-down symmetry - for example, one lever is not closed, and the lever height or brightness is visually inconsistent. Inspired by the concepts of symmetry detection of building facade structures and symmetry destruction of tumor areas in medical images, this invention introduces "symmetry consistency destruction" into lever state recognition. By extracting visual geometric features such as edges, contours, reflections, and textures of the left and right / top and bottom regions, a symmetry error index is constructed. The symmetry differences between regions are used to judge anomalies, enabling direct identification of abnormal lever states without sample training. This greatly improves the recognition system's versatility, lightweightness, and adaptability to complex scenarios.

[0129] Furthermore, the component status joint judgment module is used to perform a logical joint judgment on the component status by combining the preliminary recognition result and the secondary recognition result, and output the special equipment status recognition result; the specific process of the logical joint judgment is as follows:

[0130] A judgment logic table is established to perform a logical joint judgment in combination with the preliminary identification result and the secondary identification result; the judgment logic table is: when the special equipment is in a state where the components should be closed and the preliminary identification result and the secondary identification result are both actually closed, it is judged to be a normal state; when the special equipment is in a state where the components should be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be an abnormal state; when the special equipment is in a state where the components should not be closed and the preliminary identification result and / or the secondary identification result are actually not closed, it is judged to be a normal state.

[0131] During the conceptualization phase, it was discovered that relying solely on a single channel (such as edge recognition or structural symmetry) would be difficult to maintain recognition stability in complex environments. For example, blurred edges or light reflections can cause recognition errors. Therefore, the present invention constructs a dual-channel redundant recognition architecture that integrates static structural recognition and dynamic symmetry analysis. This fusion of "static structural feature recognition" and "dynamic geometric consistency analysis" achieves high-precision recognition of device component states. The initial recognition phase constructs a human-chair model based on the spatial relationship between key points, identifying the device component's outline and geometric closure features. The secondary recognition phase performs redundancy verification based on symmetry breaking. These two approaches operate from completely different visual mechanisms, yet provide complementary information. When the two recognition results are consistent, the device state is confirmed to be stable. When the results are inconsistent, they can be used for safety redundancy judgment or manual review prompts, thus achieving an organic combination of interpretability and safety.

[0132] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A special equipment intelligent inspection and analysis method based on image acquisition, characterized in that: include: Collect videos of special equipment in operation and extract key information of passengers' movements in the video frames; The key action point information is detected, and the relative displacement curve, angle change curve, and passenger-equipment contact relationship curve of the key action points in N consecutive frames are calculated to determine whether the passenger has completed the seat-taking action. The specific process is: calculating the vertical displacement of the shoulder key point and the hip key point in N consecutive frames to obtain the relative displacement curve; Calculate the angle changes between the hip key point and the knee key point and between the knee key point and the ankle key point in N consecutive frames to obtain the angle change curve; Calculate the spatial distance between the hip key point and the seat surface in N consecutive frames to obtain the passenger-equipment contact relationship curve; When the displacement change amplitude in the relative displacement curve is less than a preset displacement change threshold, the angle in the angle change curve is stable within the preset device standard sitting angle range, the spatial distance in the passenger-device contact relationship curve is less than a preset contact threshold, and the duration exceeds a preset time, it is determined that the seating action is completed; If it is determined that the seating action is completed, the corresponding device is determined based on the spatial relationship between the passenger and the device and marked as a device to be identified; static structural features of the device to be identified are extracted, and the component status of the device to be identified is preliminarily identified to obtain a preliminary identification result; the static structural features include component curvature and endpoint symmetric position; Based on the symmetry center of each device to be identified, the components are divided into left and right regions and upper and lower regions; dynamic geometric features of the component images in the left and right regions and upper and lower regions are extracted, and the component status of the device to be identified is secondary identified to obtain a secondary identification result; the dynamic geometric features include component outline, edge density, reflective brightness and texture features; Combine the preliminary identification results and the secondary identification results to perform a logical joint judgment on the component status and output the special equipment status identification results.

2. The method for intelligent inspection and analysis of special equipment based on image acquisition according to claim 1, characterized in that: The process of extracting the passenger's action key point information includes: Based on the deep learning human posture estimation algorithm, human body detection and key point positioning are performed on passengers in the video frames, and the passenger's action key points and corresponding key point coordinate information are extracted; the action key points include: shoulder key points, hip key points, knee key points and ankle key points; the key point coordinate information is filtered to obtain stable key point coordinates; the action key points and the stable key point coordinates are combined to obtain action key point information.

3. The method for intelligent inspection and analysis of special equipment based on image acquisition according to claim 1, characterized in that: The specific process of the preliminary identification is as follows: Based on the passenger's key motion point information and the device's geometric dimensions, a three-dimensional spatial relationship model between the person and the device is constructed; and the spatial coordinate information of the device is determined using the three-dimensional spatial relationship model between the person and the device. Based on the result of determining whether the passenger has completed the seat-taking action and the spatial coordinate information of the device, it is determined whether each device is in a state where the components should be closed; the device in the state where the components should be closed is marked as a device to be identified; In the image area of ​​the device to be identified, identifying component contours based on an edge contour extraction algorithm; extracting static structural features of the device to be identified based on the component contours; A preliminary identification is performed on the component status of the device to be identified based on the static structural features to obtain a preliminary identification result.

4. The method for intelligent inspection and analysis of special equipment based on image acquisition according to claim 1, characterized in that: The specific process of the secondary identification is: Dividing the components of each device to be identified into left and right regions and upper and lower regions according to the symmetry center; extracting dynamic geometric features of the component images of the left and right regions and the upper and lower regions; Based on the dynamic geometric features, symmetry indicators are calculated, including component contour symmetry error, edge density consistency, reflective brightness offset and texture consistency; based on the symmetry indicators, the similarity between symmetrical areas is calculated, and by comparing the similarities, the structural symmetry of the components of the device to be identified is secondary identified to obtain a secondary identification result.

5. The method for intelligent inspection and analysis of special equipment based on image acquisition according to claim 1, characterized in that: The specific process of the logical joint judgment is: Establishing a judgment logic table to perform a logical joint judgment based on the preliminary recognition result and the secondary recognition result; the judgment logic table is as follows: when the special equipment is in a state where the component should be closed and both the preliminary recognition result and the secondary recognition result are actually closed, it is determined to be a normal state; When the special equipment is in a state where the components should be closed and the initial identification result and / or the secondary identification result is that it is actually not closed, it is determined to be an abnormal state; When the special equipment is in a state where its components should not be closed, it is judged to be in a normal state.

6. An intelligent inspection and analysis system for special equipment based on image acquisition, characterized in that: Executing the special equipment intelligent inspection and analysis method based on image acquisition as claimed in claim 1, comprising: The action key point extraction module is used to collect videos of special equipment in operation and extract the action key point information of passengers in the video frames; A seating determination module is used to detect the key point information of the action, calculate the relative displacement curve, angle change curve and passenger-equipment contact relationship curve of the key points of the action in N consecutive frames, and determine whether the passenger has completed the seating action; A component status preliminary identification module is configured to, if it is determined that the seating action is completed, determine the corresponding device based on the spatial relationship between the passenger and the device and mark it as a device to be identified; extract static structural features of the device to be identified, perform preliminary identification of the component status of the device to be identified, and obtain a preliminary identification result; A component status secondary recognition module is used to divide the components into left and right regions and upper and lower regions based on the symmetry center of each device to be recognized; extract dynamic geometric features of the component images in the left and right regions and upper and lower regions, perform secondary recognition on the component status of the device to be recognized, and obtain secondary recognition results; The component status joint judgment module is used to perform logical joint judgment on the component status based on the preliminary recognition result and the secondary recognition result, and output the special equipment status recognition result.

Images